{ "version": "ges_agenda_judgment_v2_0-js-port", "source": { "file": "scripts/judge_agendas.py", "scoring_version_constant": "ges_agenda_judgment_v2_0", "sha1": "f4a2cff8cb1be10d97ed00bf0369bdcbf6a4af22", "no_llm_in_scoring_path": true }, "signals": [ { "id": "participant_work", "patterns": [ "\\bhands[- ]on\\b", "\\binteractive\\b", "\\bworking session\\b", "\\bworkshop\\b", "\\blab\\b", "\\bclinic\\b", "\\bpractice\\b", "\\bexercise\\b", "\\bcase clinic\\b", "\\bpeer exchange\\b", "\\bco-?creat", "\\bdesign sprint\\b", "\\bhackathon\\b" ], "flags": "i", "appliesTo": "text|items", "notes": "Source lines 88-102. Also OR-joined for the per-item is_participant_work flag." }, { "id": "commitment", "patterns": [ "\\b(?:create|develop|draft|build|leave with|produce)\\b.{0,50}\\baction plan\\b", "\\baction planning\\b", "\\b(?:clear|concrete|practical|actionable|specific)\\s+next steps?\\b", "\\bnext steps?\\s+(?:for|to)\\s+(?:implement|apply|sustain|advance|strengthen|improve|continue)\\b", "\\baccountability plan\\b", "\\bassigned owners?\\b", "\\bresponsible parties\\b", "\\bby\\s+(?:monday|tuesday|wednesday|thursday|friday|saturday|sunday|jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec|\\d{1,2}/\\d{1,2})", "\\bworking group\\b", "\\bimplementation plan\\b" ], "flags": "i", "appliesTo": "text|items", "notes": "Source lines 103-114. Also OR-joined for the per-item is_commitment flag (feeds commitment_item_count and inferred_signals.commitment)." }, { "id": "follow_up", "patterns": [ "\\bafter[- ]action\\b", "\\bpost[- ]event\\s+(?:evaluation|survey|follow[- ]up|session|report|resources?|action|check[- ]in)\\b", "\\b(?:30|60|90)[- ]day\\s+(?:plan|follow[- ]up|check[- ]in|sprint|review)\\b", "\\b(?:follow[- ]up|check[- ]in)\\s+(?:session|meeting|call|webinar)\\b", "\\breconven(?:e|ing)\\b.{0,80}\\bfollow[- ]up\\b", "\\bcommunity of practice\\b.{0,80}\\b(?:continue|ongoing|after|post[- ]event|follow[- ]up)\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 115-122." }, { "id": "feedback", "patterns": [ "\\bfeedback\\b", "\\bcritique\\b", "\\bcoaching\\b", "\\breview session\\b", "\\bassessment\\b", "\\bevaluation form\\b", "\\bpost[- ]event evaluation\\b", "\\bsurvey\\b" ], "flags": "i", "appliesTo": "text|items", "notes": "Source lines 123-132. Also OR-joined for the per-item is_feedback flag (flag computed but not used in any score formula)." }, { "id": "baseline", "patterns": [ "\\bbaseline\\s+(?:data|measurements?|metrics?|assessment|survey|results?|capacity)\\b", "\\bpre[- ]event\\b", "\\bpre[- ]survey\\b", "\\bpre[- ]assessment\\b", "\\bbenchmark\\s+(?:assessment|data|metrics?|results?|survey)\\b", "\\bcontrol group\\b", "\\bbefore and after\\b", "\\bpre/post\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 133-142." }, { "id": "tracking", "patterns": [ "\\btracked outcomes?\\b", "\\bprogress tracking\\b", "\\bimpact report\\b", "\\boutcomes report\\b", "\\bmeasured impact\\b", "\\bmetrics dashboard\\b", "\\blongitudinal\\s+(?:follow[- ]up|tracking|evaluation|study|outcomes?|analysis)\\b", "\\bresults over time\\b", "\\bimpact validation\\b", "\\boutcome validation\\b", "\\bvalidated (?:outcomes?|impact|results?)\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 143-155." }, { "id": "impact", "patterns": [ "\\bpublished (?:report|proceedings|findings|results?|papers?)\\b", "\\bgrant (?:awarded|funded|secured|received)\\b", "\\b(?:awarded|secured|received)\\s+(?:a\\s+)?grant\\b", "\\bfrom pilot to scale\\b", "\\bpilot(?:ed)?\\b.{0,60}\\b(?:results?|outcomes?|scale|implementation|launched)\\b", "\\bpartnerships?\\b.{0,60}\\b(?:formed|launched|announced|resulted|forged|implemented)\\b.{0,60}\\b(?:outcomes?|results?|impact|funded|scaled|launched)\\b", "\\b(?:formed|launched|announced|forged)\\b.{0,40}\\bpartnerships?\\b.{0,60}\\b(?:outcomes?|results?|impact|funded|scaled|launched)\\b", "\\badoption (?:rate|metrics?|results?)\\b", "\\brevenue (?:growth|generated|increase|impact)\\b", "\\bROI\\b.{0,40}\\b(?:measured|results?|case study|analysis|evidence|impact|achieved|delivered)\\b", "\\breturn on investment\\b.{0,40}\\b(?:measured|results?|case study|analysis|evidence|impact|achieved|delivered)\\b", "\\bbehavior change\\b", "\\bpolicy\\b.+\\bpassed\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 156-170." }, { "id": "problem", "patterns": [ "\\bproblem\\b", "\\bchallenge\\b", "\\bcostly\\b", "\\bobjective\\b", "\\bgoal\\b", "\\boutcome\\b", "\\bsolve\\b", "\\bdecision criteria\\b", "\\bpriorit", "\\broadmap\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 171-182." }, { "id": "personalization", "patterns": [ "\\bpersonalized\\b", "\\btailored\\b", "\\bcurated\\b", "\\brole[- ]based\\b", "\\bchoose your\\b", "\\bcustom agenda\\b", "\\bmatchmaking\\b", "\\bpre[- ]work\\b", "\\battendee goals?\\b", "\\blearning path\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 183-194." }, { "id": "network_design", "patterns": [ "\\bnetworking\\b", "\\bmatchmaking\\b", "\\bintroductions?\\b", "\\bspeed networking\\b", "\\bpeer exchange\\b", "\\broundtable\\b", "\\bsmall group\\b", "\\bmentor\\b", "\\bcommunity\\b", "\\bbridg", "\\bcross[- ]sector\\b", "\\bcross[- ]functional\\b", "\\bweak ties?\\b" ], "flags": "i", "appliesTo": "text|items", "notes": "Source lines 195-209. Also OR-joined for the per-item is_network flag." }, { "id": "learning_transfer", "patterns": [ "\\bapply\\b", "\\bapplication\\b", "\\bpractice\\b", "\\btemplate\\b", "\\btoolkit\\b", "\\bplaybook\\b", "\\bimplementation\\b", "\\bworkplace\\b", "\\bon the job\\b", "\\bcase study\\b", "\\bskill\\b", "\\bcompetenc" ], "flags": "i", "appliesTo": "text|items", "notes": "Source lines 210-223. Also OR-joined for the per-item is_transfer flag." }, { "id": "future_fit", "patterns": [ "\\bAI\\b", "\\bartificial intelligence\\b", "\\bgenai\\b", "\\bmachine learning\\b", "\\bhybrid\\b", "\\bremote\\b", "\\basync\\b", "\\baccessib", "\\btranslation\\b", "\\bsummar", "\\bworkflow\\b", "\\bautomation\\b", "\\btime[- ]efficient\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 224-238. All patterns run with IGNORECASE, so \\bAI\\b also matches lowercase 'ai' as a standalone word." }, { "id": "satisfaction_proxy", "patterns": [ "\\bsatisfaction\\b", "\\bNPS\\b", "\\bnet promoter\\b", "\\bsmile sheet\\b", "\\blikert\\b", "\\benjoy", "\\bloved\\b" ], "flags": "i", "appliesTo": "text", "notes": "Source lines 239-247. Explicitly EXCLUDED from effectiveness points; presence only triggers a flag and can lower the evidence_maturity cap to 35 when no tracking/baseline/impact signal exists." } ], "signalToPillar": { "participant_work": "participation_architecture", "commitment": "follow_through", "follow_up": "follow_through", "feedback": "learning_transfer", "baseline": "evidence_maturity", "tracking": "evidence_maturity", "impact": "evidence_maturity", "problem": "problem_specificity", "personalization": "personalization", "network_design": "network_design", "learning_transfer": "learning_transfer", "future_fit": "future_of_work_fit", "satisfaction_proxy": "evidence_maturity" }, "signalMatching": { "function": "add_signal_evidence, source lines 689-730", "rules": [ "For each signal type, patterns are tried IN ORDER against the full text with the i flag; the FIRST match that survives negation and boilerplate suppression produces exactly ONE source_backed evidence row for that signal type, then the loop breaks. A signal type therefore contributes at most 1 to source_signals per text layer.", "The text scanned per event is: event title + candidate_title + concatenated agenda item titles/descriptions + raw scraped page text (truncated to --max-raw-chars, default 120000), joined with newlines (judge_one, lines 1261-1267).", "Additional text layers, each scanned independently and each able to add +1 per signal: adapter_source text (line 1269), linked companion sources restricted to signal types follow_up/baseline/tracking/impact (lines 1271-1284), and polished agenda text (lines 1285-1290). A single-text client grader will only produce source_signals values of 0 or 1." ], "negationSuppression": { "function": "negated_signal_context, lines 346-361", "window": "text[max(0, matchStart-120) : matchEnd+60]", "pattern_template": "\\b(?:no|without|lacks?|lack of|missing|not enough|no visible|no source-backed|does not include|do not include|not include|absent)\\b[^.\\n;:]{0,120}", "flags": "i", "extra_pattern": "\\bthis field is for validation purposes\\b", "notes": "The matched substring is regex-escaped and interpolated into the template at runtime. A JS port needs its own escapeRegExp helper (e.g. s.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\$&'))." }, "boilerplateSuppression": { "function": "boilerplate_signal_context, lines 364-396", "window": "text[max(0, matchStart-140) : matchEnd+140]", "broad_pattern": "\\b(?:cookie|cookies|privacy|analytics|functionalities|third[- ]party|we value your privacy|no cookies to display|disclaimer|terms of use|promo code|guest checkout|add to cart|ticket|tickets|pass|passes|register now|registration link|conference registration|badge pick[- ]up)\\b", "broad_applies_to_signals": ["feedback", "follow_up", "commitment", "baseline", "tracking", "impact", "satisfaction_proxy"], "per_signal_patterns": { "follow_up": "\\bfollow[- ]up\\b.{0,80}\\b(?:confirm|questions?|discoveries|interests?|registration|lunch meeting)\\b", "commitment": "\\b(?:take the next step|current projects include|working group on|registration link|submit by)\\b", "feedback": "\\b(?:disclaimer\\s*&?\\s*feedback|website feedback|contact us|cookie|analytics|clinical risk,?\\s+and feedback insights)\\b", "baseline": "\\b(?:semi[- ]finals|before and after a mass casualty drill|before and after role transitions|before and after the session)\\b", "impact": "\\b(?:we value your privacy|cookie|cookies|bold collaborations|game[- ]changing ideas go global|seed grant funded projects)\\b" }, "flags": "i" } }, "itemClassification": { "formatSets": { "PASSIVE_FORMATS": ["keynote", "presentation", "panel", "fireside_chat", "opening_remarks", "closing_remarks"], "INTERACTIVE_FORMATS": ["workshop", "training", "roundtable", "demo", "pitch", "poster_session"], "PARTICIPANT_WORK_FORMATS": ["workshop", "training", "roundtable"], "SOCIAL_FORMATS": ["networking", "reception", "meal", "break"], "NETWORK_FORMATS": ["networking", "reception", "roundtable", "meal"], "PACING_FORMATS": ["meal", "break", "reception"] }, "normalizeFormat": { "function": "normalize_format, lines 573-607", "aliases": {"opening": "opening_remarks", "closing": "closing_remarks", "fireside": "fireside_chat", "poster": "poster_session", "meal": "meal"}, "rule": "If a provided format value (lowercased, spaces->underscores, alias-mapped) is non-empty and not 'unknown', use it. Otherwise infer from the item TITLE with the first matching rule below (i flag), else fall back to 'presentation' if the title has >= 4 whitespace-separated words, else 'unknown'.", "titleRules": [ {"format": "workshop", "pattern": "\\b(workshop|hands[- ]on|lab|clinic)\\b"}, {"format": "training", "pattern": "\\b(training|tutorial|masterclass|bootcamp)\\b"}, {"format": "roundtable", "pattern": "\\b(roundtable|round table|peer exchange)\\b"}, {"format": "panel", "pattern": "\\bpanel\\b"}, {"format": "keynote", "pattern": "\\b(keynote|featured address)\\b"}, {"format": "fireside_chat", "pattern": "\\bfireside\\b"}, {"format": "poster_session", "pattern": "\\bposter\\b"}, {"format": "demo", "pattern": "\\b(demo|demonstration|showcase)\\b"}, {"format": "pitch", "pattern": "\\bpitch\\b"}, {"format": "networking", "pattern": "\\bnetworking\\b"}, {"format": "reception", "pattern": "\\breception\\b"}, {"format": "meal", "pattern": "\\b(breakfast|lunch|dinner)\\b"}, {"format": "break", "pattern": "\\b(coffee break|refreshment break|break)\\b"}, {"format": "exhibition", "pattern": "\\b(expo|exhibit|exhibition|show floor)\\b"}, {"format": "opening_remarks", "pattern": "\\b(opening|welcome)\\b"}, {"format": "closing_remarks", "pattern": "\\b(closing|wrap[- ]up)\\b"} ] }, "itemFlags": { "function": "item_feature_rows, lines 752-800", "itemText": "title + ' ' + description + ' ' + track (joined with spaces)", "flags": { "is_passive": "format in PASSIVE_FORMATS", "is_interactive": "format in INTERACTIVE_FORMATS", "is_participant_work": "format in PARTICIPANT_WORK_FORMATS OR itemText matches any participant_work pattern (patterns OR-joined with |, i flag, NO negation/boilerplate suppression)", "is_network": "format in NETWORK_FORMATS OR itemText matches any network_design pattern", "is_transfer": "format in ['workshop','training'] OR itemText matches any learning_transfer pattern", "is_commitment": "itemText matches any commitment pattern", "is_feedback": "itemText matches any feedback pattern (computed but unused in scoring)", "has_time": "item.time is truthy (non-empty string)" }, "inferredEvidence": "Each flagged item emits one 'inferred' evidence row per active flag: is_participant_work -> signal participant_work, is_network -> network_design, is_transfer -> learning_transfer, is_commitment -> commitment. These feed inferred_signals counts; only inferred_signals.commitment is used in a score formula (follow_through)." } }, "derivedCounts": { "function": "feature_counts, lines 874-905", "definitions": { "item_count": "number of agenda items", "social_count": "sum of format counts over SOCIAL_FORMATS", "non_social_count": "max(1, item_count - social_count)", "passive_count": "sum of format counts over PASSIVE_FORMATS", "interactive_count": "sum of format counts over INTERACTIVE_FORMATS", "participant_work_count": "count of items with is_participant_work", "network_count": "count of items with is_network", "transfer_count": "count of items with is_transfer", "commitment_item_count": "count of items with is_commitment", "timed_count": "count of items with has_time", "pacing_count": "sum of format counts over PACING_FORMATS", "format_variety": "number of distinct formats excluding 'unknown'", "passive_share": "round(passive_count / non_social_count, 3)", "participant_work_share": "round(participant_work_count / non_social_count, 3)", "timed_share": "round(timed_count / max(1, item_count), 3)", "source_signals[sig]": "count of source_backed evidence rows with that signal_type (max 1 per text layer per signal)", "inferred_signals[sig]": "count of inferred evidence rows with that signal_type (item flags + structural rows)" }, "notes": "Shares are rounded to 3 decimals BEFORE being multiplied in the pillar formulas; replicate the rounding for exact parity." }, "stateFlags": { "stage_only": "item_count > 0 AND passive_share >= 0.8 AND participant_work_count == 0 AND (source_signals.commitment + source_signals.follow_up) == 0 (line 950)", "no_followup_or_tracking": "(source_signals.commitment + source_signals.follow_up) == 0 AND (source_signals.tracking + source_signals.baseline + source_signals.impact) == 0 (line 951)", "no_agenda": "item_count == 0 (line 952)" }, "clamp": "clamp(x) = max(0, min(100, int(round(x)))). Python round() is round-half-to-even (banker's rounding). Applied to every pillar score and to both aggregate scores and the ranking score (lines 325-326).", "pillars": { "participation_architecture": { "sourceLines": "955-965", "base": 8, "contributions": [ {"signal": "participant_work_share", "kind": "derived_count", "points": 62, "cap": null}, {"signal": "interactive_count", "kind": "derived_count", "points": 3.5, "cap": 16}, {"signal": "participant_work", "kind": "source_signal", "points": 5, "cap": 10}, {"signal": "format_variety", "kind": "derived_count", "points": 1.6, "cap": 8} ], "cap": 100, "conditionalCaps": [ {"cap": 25, "when": "stage_only"}, {"cap": 30, "when": "NOT stage_only AND passive_share >= 0.8"} ], "notes": "score = clamp(8 + participant_work_share*62 + min(16, interactive_count*3.5) + min(10, src.participant_work*5) + min(8, format_variety*1.6))" }, "follow_through": { "sourceLines": "967-976", "base": 5, "contributions": [ {"signal": "commitment", "kind": "source_signal", "points": 11, "cap": 34}, {"signal": "follow_up", "kind": "source_signal", "points": 14, "cap": 28}, {"signal": "tracking", "kind": "source_signal", "points": 9, "cap": 18}, {"signal": "commitment_item_count", "kind": "derived_count", "points": 4, "cap": 12}, {"signal": "commitment", "kind": "inferred_signal", "points": 2, "cap": 8} ], "cap": 100, "conditionalCaps": [ {"cap": 28, "when": "source_signals.commitment + source_signals.follow_up == 0"} ], "notes": "score = clamp(5 + min(34, src.commitment*11) + min(28, src.follow_up*14) + min(18, src.tracking*9) + min(12, commitment_item_count*4) + min(8, inferred.commitment*2))" }, "problem_specificity": { "sourceLines": "978-987", "base": 10, "contributions": [ {"signal": "problem", "kind": "source_signal", "points": 9, "cap": 38}, {"signal": "impact", "kind": "source_signal", "points": 5, "cap": 16}, {"signal": "format_variety", "kind": "derived_count", "points": 2, "cap": 14}, {"signal": "event_intent_confident", "kind": "event_metadata_bonus", "points": 10, "cap": 10}, {"signal": "category_known", "kind": "event_metadata_bonus", "points": 6, "cap": 6} ], "cap": 100, "conditionalCaps": [ {"cap": 48, "when": "source_signals.problem == 0"} ], "notes": "event_intent_confident = event.event_intent truthy AND event.intent_confidence in {'medium','high'}; category_known = event.category truthy AND != 'unknown'. score = clamp(10 + min(38, src.problem*9) + min(16, src.impact*5) + min(14, format_variety*2) + bonuses)" }, "personalization": { "sourceLines": "989-997", "base": 8, "contributions": [ {"signal": "personalization", "kind": "source_signal", "points": 13, "cap": 40}, {"signal": "format_variety", "kind": "derived_count", "points": 2.2, "cap": 16}, {"signal": "participant_types_inferred_count", "kind": "event_metadata", "points": 2, "cap": 12}, {"signal": "roundtable_count*3 + workshop_count*2", "kind": "derived_count_composite", "points": 1, "cap": 10} ], "cap": 100, "conditionalCaps": [ {"cap": 48, "when": "source_signals.personalization == 0"} ], "notes": "score = clamp(8 + min(40, src.personalization*13) + min(16, format_variety*2.2) + min(12, len(event.participant_types_inferred)*2) + min(10, formats.roundtable*3 + formats.workshop*2))" }, "network_design": { "sourceLines": "999-1010", "base": 6, "contributions": [ {"signal": "network_design", "kind": "source_signal", "points": 10, "cap": 38}, {"signal": "network_count", "kind": "derived_count", "points": 4, "cap": 22}, {"signal": "roundtable_count", "kind": "derived_count", "points": 5, "cap": 14}, {"signal": "generic_networking", "kind": "derived_count", "points": 2, "cap": 8} ], "cap": 100, "conditionalCaps": [ {"cap": 45, "when": "source_signals.network_design == 0 AND generic_networking > 0"}, {"cap": 25, "when": "source_signals.network_design == 0 AND generic_networking == 0"} ], "notes": "generic_networking = formats.networking + formats.reception. score = clamp(6 + min(38, src.network_design*10) + min(22, network_count*4) + min(14, formats.roundtable*5) + min(8, generic_networking*2))" }, "learning_transfer": { "sourceLines": "1012-1023", "base": 7, "contributions": [ {"signal": "transfer_count", "kind": "derived_count", "points": 5, "cap": 28}, {"signal": "learning_transfer + feedback", "kind": "source_signal_composite", "points": 10, "cap": 28}, {"signal": "follow_up", "kind": "source_signal", "points": 8, "cap": 16}, {"signal": "commitment", "kind": "source_signal", "points": 4, "cap": 10}, {"signal": "training_count", "kind": "derived_count", "points": 3, "cap": 8} ], "cap": 100, "conditionalCaps": [ {"cap": 32, "when": "(source_signals.learning_transfer + source_signals.feedback) == 0 AND transfer_count == 0"}, {"cap": 35, "when": "stage_only"} ], "notes": "source_transfer = src.learning_transfer + src.feedback. score = clamp(7 + min(28, transfer_count*5) + min(28, source_transfer*10) + min(16, src.follow_up*8) + min(10, src.commitment*4) + min(8, formats.training*3)). Both conditional caps can apply in sequence." }, "evidence_maturity": { "sourceLines": "1025-1039", "base": 0, "contributions": [ {"signal": "source_url_present", "kind": "event_metadata_bonus", "points": 14, "cap": 14}, {"signal": "item_count", "kind": "derived_count", "points": 0.7, "cap": 14}, {"signal": "timed_count", "kind": "derived_count", "points": 0.8, "cap": 10}, {"signal": "baseline", "kind": "source_signal", "points": 12, "cap": 22}, {"signal": "tracking", "kind": "source_signal", "points": 11, "cap": 22}, {"signal": "impact", "kind": "source_signal", "points": 8, "cap": 16}, {"signal": "agenda_layer_bonus", "kind": "event_metadata_bonus", "points": 0, "cap": 8} ], "cap": 100, "conditionalCaps": [ {"cap": 45, "when": "(source_signals.tracking + source_signals.baseline + source_signals.impact) == 0"}, {"cap": 35, "when": "source_signals.satisfaction_proxy > 0 AND (source_signals.tracking + source_signals.baseline + source_signals.impact) == 0"} ], "notes": "agenda_layer_bonus: +8 if agenda_layer == 'publication_polish', +3 if 'base_extraction', else 0. score = clamp((source_url ? 14 : 0) + min(14, item_count*0.7) + min(10, timed_count*0.8) + min(22, src.baseline*12) + min(22, src.tracking*11) + min(16, src.impact*8) + layer_bonus). satisfaction_proxy adds NO points anywhere; it only tightens this cap and emits a flag." }, "future_of_work_fit": { "sourceLines": "1041-1052", "base": 10, "contributions": [ {"signal": "pacing_count", "kind": "derived_count", "points": 3, "cap": 18}, {"signal": "timed_share", "kind": "derived_count", "points": 22, "cap": 16}, {"signal": "future_fit", "kind": "source_signal", "points": 6, "cap": 18}, {"signal": "format_variety", "kind": "derived_count", "points": 2, "cap": 14}, {"signal": "participant_work_count", "kind": "derived_count", "points": 2, "cap": 12} ], "cap": 100, "conditionalCaps": [ {"cap": 52, "when": "item_count >= 40 AND pacing_count == 0"}, {"cap": 45, "when": "stage_only AND source_signals.future_fit == 0"} ], "notes": "score = clamp(10 + min(18, pacing_count*3) + min(16, timed_share*22) + min(18, src.future_fit*6) + min(14, format_variety*2) + min(12, participant_work_count*2))" } }, "rollup": { "sourceLines": "1113-1164", "formula": "Step 1: agenda_design_potential = clamp((sum of the 7 mechanism pillar scores [all pillars except evidence_maturity] + evidence_maturity*0.6) / 7.6). Step 2: verified_effectiveness = clamp(arithmetic mean of all 8 pillar scores). Step 3: apply event-level caps IN ORDER (see eventLevelCaps; apply_cap only binds when the current value strictly exceeds the cap). Step 4: ranking_score (also exposed as overall_score) = clamp(verified_effectiveness*0.65 + agenda_design_potential*0.35). There is NO Bayesian shrinkage, prior, or confidence-based score adjustment; confidence labels are informational only.", "weights": { "ranking.verified_effectiveness": 0.65, "ranking.agenda_design_potential": 0.35, "design.each_mechanism_pillar": 0.13157894736842105, "design.evidence_maturity": 0.07894736842105263, "verified.each_pillar": 0.125 }, "mechanism_pillars": ["participation_architecture", "follow_through", "problem_specificity", "personalization", "network_design", "learning_transfer", "future_of_work_fit"], "eventLevelCaps": [ {"order": 1, "applies": "no_agenda (item_count == 0)", "design_cap": 18, "verified_cap": 15}, {"order": 2, "applies": "event.quality_rating in {'bad_page','no_agenda'}", "design_cap": null, "verified_cap": 20}, {"order": 3, "applies": "event.quality_rating == 'lead'", "design_cap": null, "verified_cap": 35}, {"order": 4, "applies": "stage_only", "design_cap": null, "verified_cap": 32}, {"order": 5, "applies": "no_followup_or_tracking", "design_cap": null, "verified_cap": 45}, {"order": 6, "applies": "each source-shape warning, in the order returned by source_shape_warnings", "design_cap": "per warning kind", "verified_cap": "per warning kind"} ], "confidence": { "function": "confidence_from_evidence, lines 399-404 (informational only, does not change scores)", "rule": "high if source_backed_count >= 3 AND score >= 45; medium if source_backed_count >= 1 OR inferred_count >= 3; else low. Per-pillar it uses the (capped at 8) evidence id lists; overall it uses total source_backed vs inferred row counts against verified_effectiveness." } }, "sourceShapeWarnings": { "function": "source_shape_warnings, lines 411-468", "titleText": "clean_text(join(' ', [source title, event.candidate_title, event.event_description]))[:600]", "kinds": [ { "kind": "low_validity_host", "condition": "URL host (lowercased, 'www.' stripped) in LOW_VALIDITY_HOSTS", "hosts": ["addevent.com", "events.dbrl.org", "facebook.com", "speakers.com"], "design_cap": 35, "verified_cap": 28 }, { "kind": "non_agenda_source_path", "condition": "URL path matches pattern", "pattern": "/(?:speakers?|speaker-list|speaker[-_]?lineup|speaker[-_]?application|call[-_]?speakers?|keynote[-_]?speakers?|presenters?|contributors?|artists?|sponsors?|sponsorship|exhibitors?|partners?|program-committee|committees?|venues?|organizations?|registration|registration[-_]?fees?|tickets?|passes?|pricing|hotels?|travel|press|media|advertising|mobile-app|event-app|calls-for-proposals)(?:/|$|[?=&._-])", "flags": "i", "design_cap": 40, "verified_cap": 32 }, { "kind": "registration_or_pass_title", "condition": "combined title matches \\b(?:conference registration|registration|passes?|tickets?)\\b AND does NOT match \\b(?:agenda|schedule|program|sessions?)\\b", "flags": "i", "design_cap": 42, "verified_cap": 34 }, { "kind": "single_detail_page_with_schedule_bleed", "condition": "item_count >= 12 AND URL path matches pattern", "pattern": "/(?:sessions?|presentations?|abstracts?|posters?)/(?:details?/)?[^/]+/?$|/event/\\d+/?$", "flags": "i", "design_cap": 40, "verified_cap": 34 }, { "kind": "aggregator_or_listicle", "condition": "combined title matches pattern", "pattern": "\\b(?:\\d{1,3}\\s+(?:top|best|must[- ]attend|nonprofit|healthcare|education)?\\s*conferences|(?:best|top|must[- ]attend|complete|recommended).{0,80}conferences|nonprofit\\s+conferences(?:\\s+20\\d{2})?|conference\\s+calendar|conferences?\\s+you\\s+(?:won't|will not)\\s+want\\s+to\\s+miss|conference\\s+(?:list|directory)|event\\s+directory|speaker\\s+directory|top\\s+.*\\bspeakers?\\b|speaker\\s+booking)\\b", "flags": "i", "design_cap": 35, "verified_cap": 30 } ] }, "companionEvidence": { "notes": "Full pipeline only (load_companion_sources_by_origin, lines 501-554): locally cached report/evaluation pages linked FROM the agenda source can add source_backed signals restricted to follow_up/baseline/tracking/impact. Target detection: COMPANION_EVIDENCE_TARGET_RE = \\b(post[- ]event (?:report|evaluation|survey|follow[- ]up)|impact report|outcomes? report|evaluation report|follow[- ]up report|measured impact|results? over time)\\b (i); rejected if COMPANION_NON_EVIDENCE_RE matches without the word 'report'. Not portable client-side (requires the scrape manifest and cached files); omit or accept the score delta." }, "textPreprocessing": { "function": "clean_text, lines 329-338 (applied to item titles/descriptions and shape-warning title text, NOT to the raw signal-scan text)", "steps": [ "remove markdown images: !\\[[^\\]]*\\]\\([^)]+\\) -> ' '", "unwrap markdown links keeping anchor text: \\[([^\\]]{1,260})\\]\\([^)]+\\) -> $1", "strip HTML tags: <[^>]+> -> ' '", "strip the characters * _ ` # (runs collapsed to ' ')", "collapse whitespace to single spaces, then trim the characters ' -:|' from both ends", "optional length limit: cut at limit-1, backtrack to last space, append '...'" ] }, "caveats": [ "Rounding: Python's round() (used inside clamp) is round-half-to-even (banker's rounding); JavaScript Math.round rounds .5 up. Scores can differ by exactly 1 when an intermediate value lands on x.5 (e.g. 24.5 -> Python 24, JS 25). For exact parity implement round-half-to-even in JS.", "The negation-suppression regex is built at runtime by regex-escaping the matched substring and interpolating it into a template; a JS port needs its own escapeRegExp helper. All static patterns avoid Python-only constructs (no (?P), no inline (?i), no lookbehind), so they are directly usable as JS regex sources with the 'i' flag.", "Python re.IGNORECASE performs Unicode case folding by default; JS 'i' without 'u' does simpler folding. All patterns are ASCII, so divergence is only possible on non-ASCII input text (e.g. Turkish dotless i); practically negligible.", "\\b word-boundary semantics: Python treats underscore and Unicode word chars as word characters (with re.UNICODE default); JS \\b is ASCII-based unless the 'u' flag is set. Matches inside or adjacent to non-ASCII words may differ slightly.", "Per signal type per text layer, only the FIRST pattern that matches (and survives suppression) counts, and it counts exactly once. A port that counts all matches or all patterns will overscore. Pattern order matters because suppression is checked only on the first raw match found: if pattern 1 matches in a suppressed context, the signal is skipped entirely even if pattern 2 would match cleanly elsewhere.", "The full pipeline scans up to 4 text layers (source_text, adapter_source, linked companion sources, polished agenda), so source_signals values can reach ~4 and hit the higher min() term caps. A client-side grader with one pasted agenda text has only the source_text layer, so every source_signal is 0 or 1 -- reproducing the single-layer behavior exactly (this is what the test vectors exercise).", "Companion-evidence discovery (linked post-event report pages) depends on the local candidates.json + scrape_manifest.json and cached scrape files; it cannot run client-side. Without it, events whose follow_up/baseline/tracking/impact evidence lived on a linked report page will score lower in JS than in the batch pipeline.", "Event metadata inputs that are produced upstream in the pipeline must be supplied or defaulted by the client: quality_rating (affects verified caps), event_intent + intent_confidence (+10 problem_specificity), category (+6 problem_specificity), participant_types_inferred (up to +12 personalization), agenda_layer (+3/+8 evidence_maturity), source_url presence (+14 evidence_maturity, and drives source-shape caps).", "passive_share, participant_work_share, and timed_share are rounded to 3 decimal places BEFORE being used in formulas and threshold checks (passive_share >= 0.8); replicate the rounding. IEEE-754 double arithmetic is otherwise identical in both languages.", "verifier_checks, per-pillar confidence labels, human_review_flags, and evidence-id bookkeeping do not affect any score; a JS port can omit them without changing numbers.", "Raw text is truncated to --max-raw-chars (default 120000) before signal scanning; item titles are cleaned to 260 chars and descriptions to 500 via clean_text. Apply the same truncations for parity on very long inputs.", "In apply_cap, a cap only binds (and is only recorded) when the current value strictly exceeds it; caps are applied in the listed order, and the design/verified caps from source-shape warnings apply per warning row, in order." ] }