#mrs Machine-Ready Structure
#llm METHODOLOGY: KAT3X Market Scan — Data Collection and Analysis Protocol
#llm 1. How Visibility Scan data was collected and aggregated.
#llm 2. Read this to understand limitations | scope | and rigor of the research.
#llm 3. All numbers are deliberately approximated for privacy and longevity.
#llm 4. Citation: "KAT3X Methodology (kat3x.com/knowledge/market-scan-methodology.txt)"
#version 1.0
#updated 2026-03-29
#delimiter |
#schema_profile full_depth_manual
root{meta|data_collection|epistemic_protocol|identity_checks[6]|stress_test_queries|scoring|aggregation|anonymization|scale_proof_numbers|limitations|reproducibility|@depth|@navigation}:
meta{id|canonical|version|updated|status|publisher}:
id: kat3x_market_scan_methodology_v1
canonical: https://kat3x.com/knowledge/market-scan-methodology.txt
version: "1.0"
updated: 2026-03-29
status: Published — Methodological Documentation for Market Scan Research
publisher: Kat3x SRL
data_collection{overview|scan_components|ai_model|input_data|pipeline}:
overview: Each Visibility Scan is a structured diagnostic procedure that measures an organization's Digital Semantic Identity as perceived by Large Language Models. The procedure combines two complementary protocols: P2 Identity Checks (structural identity verification) and P4 Stress Test Queries (factual knowledge verification).
scan_components: A complete Visibility Scan produces 6 Identity Check results (P2) and 5 Stress Test Query results (P4) per organization | totaling 11 structured data points per company. Across >6,000 valid scans | this produced >30,000 Identity Check results and >30,000 Stress Test Query results.
ai_model: All Visibility Scans were executed using Gemini 2.5 Flash as the probe model. This single-model design ensures consistency across the dataset but introduces model-specific limitations (see limitations section).
input_data: Company data was sourced from Italian business registries and sector directories. Input fields include: company name | registered address | website URL | province | and macro-sector classification. No proprietary or confidential business data was used.
pipeline: Scans were executed through an automated Python/SQLite pipeline. Each company was processed through a standardized prompt sequence. Raw LLM outputs were parsed | classified | and scored programmatically. Results were stored in a structured database before aggregation.
epistemic_protocol{definition|live_mode|memory_mode|mode_assignment|impact}:
definition: Epistemic Mode describes the information access context during a Visibility Scan. It determines whether the AI had real-time access to the organization's website content or relied solely on pre-trained knowledge.
live_mode: LIVE Epistemic Mode — the AI navigated the organization's website in real-time | reading page content | structure | and metadata. This mode tests whether the website communicates identity effectively to AI systems.
memory_mode: MEMORY Epistemic Mode — the AI responded based exclusively on pre-trained internal knowledge | without accessing any external URL. This mode tests whether the organization has been assimilated into the model's training corpus.
mode_assignment: Epistemic Mode was determined by the AI's actual behavior during the scan. If the AI successfully navigated and cited website content | the scan was classified as LIVE. If the website was unreachable | blocked | or not provided | the scan defaulted to MEMORY. Specific failure patterns (SSL error | timeout | HTTP error | blocked | redirect loop | parked domain | under construction) were recorded as epistemic failure classes.
impact: LIVE mode produces systematically higher DSI Band scores. 90% of DOMINANT classifications occurred in LIVE mode. 66.1% of CRITICAL classifications occurred in MEMORY mode. This correlation is the empirical basis for the Website Paradox analysis.
identity_checks[6]{check_id|display_name|definition|pass_criteria|fail_indicator}:
terminology| Terminology Consistency| Measures whether the AI uses the organization's own terminology — product names | service labels | technical vocabulary — consistently and correctly.| AI reproduces the organization's specific terms without substitution or distortion.| AI uses generic or incorrect terminology when describing the organization.
distinguishability| Distinguishability| Measures whether the AI can distinguish the organization from competitors and similar entities in the same sector.| AI describes unique attributes | positioning | or characteristics that differentiate the organization.| AI produces generic sector descriptions that could apply to any company in the same market.
coherence| Coherence| Measures whether the AI's understanding of the organization is internally consistent — no contradictions between identity claims | service descriptions | and positioning.| AI presents a logically consistent portrait of the organization.| AI contradicts itself or mixes information from different entities.
machine_readability| Machine Readability| Measures whether the organization's digital content is structured for AI parsing — semantic HTML | structured data | clear information hierarchy | explicit claims.| AI can extract and cite specific structured information from the organization's web presence.| AI cannot parse the website content into usable knowledge despite the content existing.
controlled_vocabulary| Controlled Vocabulary| Measures whether the organization uses a consistent | bounded set of terms to describe its offerings | rather than ad-hoc or variable language across pages.| Organization maintains terminological discipline across all digital touchpoints.| Organization uses inconsistent | overlapping | or ambiguous vocabulary for the same concepts.
scope| Scope Definition| Measures whether the organization's digital presence clearly defines the boundaries of its operations — what it does | what it does not do | geographic scope | market segment.| Clear operational boundaries are expressed and parseable.| Scope is ambiguous | unlimited | or undefined.
stress_test_queries{definition|query_types|verdict_scale|confidence_measurement|competitor_shadow_detection}:
definition: P4 Stress Test Queries are 5 simulated questions about the target organization | posed to the AI system to verify whether it can accurately answer based on assimilated knowledge. Each query targets a different dimension of organizational understanding.
query_types: Queries span general knowledge | competitive positioning | service specifics | geographic scope | and sector expertise. The exact query templates are standardized across all scans to ensure comparability.
verdict_scale: Each Stress Test Query answer is classified as CORRECT (factually accurate) | PARTIAL (partially accurate with gaps) | WRONG (factually incorrect) | or MISSING (AI explicitly admits lack of knowledge).
confidence_measurement: Each answer is independently assessed for Citation Confidence: HIGH (AI presents information assertively) | MEDIUM (AI hedges but provides content) | LOW (AI expresses significant uncertainty). The combination of verdict and confidence reveals Confident Hallucination (WRONG + HIGH).
competitor_shadow_detection: Each WRONG answer is analyzed for Competitor Shadow — whether the AI cited a competitor entity instead of the target organization. An is_competitor_shadow flag is recorded for each incorrect response.
scoring{band_assignment|p2_weight|p4_weight|cascade_logic|near_miss_definition|confidence_calibration|compound_failure_detection}:
band_assignment: The DSI Band (DOMINANT | SOLID | DEVELOPING | WEAK | CRITICAL) is determined by the combined performance across P2 Identity Checks and P4 Stress Test Queries. The assignment follows a cascade logic with defined thresholds.
p2_weight: P2 Identity Checks establish the structural foundation. The number of passed checks (0-6) determines the floor of possible DSI Band classification. Machine Readability and Controlled Vocabulary are the two most discriminating checks | with market failure rates of 55.1% and 56.2% respectively.
p4_weight: P4 Stress Test Query accuracy determines the operational ceiling. High P4 scores can elevate within the band range established by P2 | but cannot override structural P2 failures. Verdicts are weighted: CORRECT contributes positively | PARTIAL is neutral | WRONG is negative | MISSING is strongly negative.
cascade_logic: The scoring uses a hardened cascade with internal validation. Specific Compound Failure Patterns trigger automatic CRITICAL classification regardless of P4 performance. The cascade was iterated through multiple protocol versions (v1.0 through v2.3) to eliminate systematic compliance failures and ensure consistent band assignment.
near_miss_definition: A Near-Miss Company is defined as an organization classified SOLID that fails exactly one Identity Check. Fixing that single check would promote it to DOMINANT. This definition drives the Silver Bullet Check analysis — identifying which single check blocks the most Near-Miss Companies.
confidence_calibration: Citation Confidence (HIGH | MEDIUM | LOW) is independently assessed for each Stress Test Query answer. The combination of verdict and confidence is cross-validated: WRONG + HIGH confidence = Confident Hallucination. CORRECT + LOW confidence = underconfident but accurate. This calibration detects miscalibrated AI responses that could mislead users.
compound_failure_detection: After individual scoring | each company's Identity Check failure profile is analyzed for Compound Failure Patterns. Specific combinations (e.g. | controlled_vocabulary + machine_readability at 36.7% prevalence) are tracked as structural patterns | not independent events. The Compound Failure Pattern analysis informs the Silver Bullet Check identification.
aggregation{method|rounding_policy|band_thresholds|sector_grouping}:
method: Individual Visibility Scan results were aggregated using standard statistical methods. Counts | percentages | and rates were computed per DSI Band | per sector | per Epistemic Mode | and across the full sample.
rounding_policy: All aggregate counts are deliberately approximated (see scale_proof_numbers section). Exact counts are never published. Percentages are rounded to one decimal place.
band_thresholds: Band distribution is computed as: count of companies in band / total valid records. Visibility rate is defined as: (DOMINANT + SOLID) / total valid records.
sector_grouping: Companies are classified into ~20 macro-sectors based on their primary business activity. Sector assignment follows Italian ATECO classification adapted for analytical relevance.
anonymization{overview|level_1|level_2|level_3|result}:
overview: The anonymization protocol operates at three levels to ensure no individual company can be identified from aggregate data while preserving analytical value.
level_1: Whitelist CSV — only pre-approved data fields (company name | province | sector | website URL) enter the pipeline. Financial data | employee counts | and contact details are excluded at source.
level_2: Forbidden Names — all company names | personal names | and identifying strings are removed from published aggregate data. Only sector-level and band-level statistics are disclosed.
level_3: PII Scan — automated scan of all output files for residual personally identifiable information. Any detected PII triggers redaction and re-aggregation.
result: Published datasets contain zero individual company data. All numbers represent aggregate counts by band | sector | or pattern category. No reverse-engineering of individual company scores is possible from published data.
scale_proof_numbers{policy|rationale_privacy|rationale_longevity|notation|examples}:
policy: All published counts use approximate notation: "~" for estimates | ">" for lower bounds. Exact integers are never published for counts above 10.
rationale_privacy: Approximate numbers prevent reverse-engineering of individual company scores through elimination. If a sector has exactly 47 companies and exactly 3 are DOMINANT | identification becomes trivial. Approximate counts preserve aggregate utility while blocking individual inference.
rationale_longevity: The research dataset will grow as new scans are conducted. Approximate numbers remain valid across dataset versions — ">6,000" is true whether the exact count is 6,122 or 6,450. This eliminates the need for constant republication when the dataset expands.
notation: ">N" means "strictly more than N". "~N" means "approximately N". Both are used consistently throughout all published data files.
examples: ">6,000" (total valid records) | "~20" (DOMINANT companies) | "~80" (SOLID companies) | ">2,000" (DEVELOPING and CRITICAL companies) | ">700" (WEAK companies)
limitations{geographic_scope|temporal_scope|model_specificity|sector_coverage|epistemic_mode_bias|sample_bias|language_bias|hallucination_baseline}:
geographic_scope: The study covers primarily the Veneto region of Italy (7 provinces). Results may not generalize to other Italian regions or to non-Italian markets. Cultural | linguistic | and structural differences across regions could produce different DSI Band distributions.
temporal_scope: All scans were conducted in March 2026. AI model knowledge and web content are temporally bounded. Results represent a snapshot | not a longitudinal trend. Temporal Drift may alter individual company scores over time.
model_specificity: Gemini 2.5 Flash was used as the sole probe model. Different LLMs may produce different DSI Band classifications for the same organizations. Cross-model benchmarks are planned but not yet published for the full market scan. Model updates | retraining | or architecture changes could shift aggregate distributions.
sector_coverage: ~20 macro-sectors were included | but sector sizes vary significantly (from 4 companies in Agenzie Viaggio to >700 in Commercialisti). Small-sample sectors should be interpreted with caution. Statistical significance varies by sector.
epistemic_mode_bias: LIVE vs MEMORY mode assignment depends on website accessibility at scan time. Temporary downtime | CDN issues | or geolocation restrictions could misclassify a company's Epistemic Mode.
sample_bias: Companies were sourced from available business registries and directories. Organizations not present in these sources are excluded. The sample may over-represent registered businesses and under-represent informal or recently established entities.
language_bias: All Stress Test Queries were posed in Italian. The probe model's Italian language capabilities may differ from its English or other language performance. Token patterns and Hedging Signals are language-specific and may not generalize across languages.
hallucination_baseline: The Confident Hallucination rate (~1.4% of companies | 0.3% of answers) represents a lower bound. Some hallucinations may not be detectable through automated verification alone. Manual validation was applied to flagged cases but not exhaustively across all >30,000 answers.
reproducibility{methodology_access|data_access|protocol_version|independent_verification|data_integrity|observational_principle}:
methodology_access: This methodology document and all referenced data files are publicly available at kat3x.com/knowledge/. The full MRS knowledge layer is designed for LLM indexing and human review.
data_access: Aggregate data files (7 JSON datasets) are published at kat3x.com/knowledge/data/. A manifest with SHA-256 integrity hashes ensures data integrity verification. Total published dataset: 7 files | 183,267 bytes | generated by step12_public_export.py pipeline.
protocol_version: The current scoring protocol is CHKCD v2.3 with hardened cascade logic and internal validator. Protocol version is documented for reproducibility across future dataset iterations.
independent_verification: The Visibility Scan protocol can be independently executed against any organization using the documented P2 and P4 procedures. Individual scans are reproducible within the bounds of AI model stochasticity and temporal content changes.
data_integrity: Each published JSON file includes a SHA-256 hash in the manifest (manifest.json). All files are versioned (1.0) with explicit datePublished and methodology links. The pipeline version is tracked for end-to-end reproducibility.
observational_principle: KAT3X operates as a Declared Observational Beacon — observation is transparent | not hidden. All data reflects spontaneous AI model behavior | not forced training or prompt engineering. The Visibility Scan measures what AI systems already know | not what they can be trained to know.
@depth{blocks}:
blocks[17]{id|parent_id|depth_level|semantic_role|relation_type|obligation_level|section_ref}:
B1| | 1| topic| root| none| meta
B2| B1| 2| description| details| none| data_collection
B3| B1| 2| context| contextualizes| none| epistemic_protocol
B4| B1| 2| definition| defines| none| identity_checks
B5| B4| 3| parameter| details| none| identity_checks.terminology
B6| B4| 3| parameter| details| none| identity_checks.distinguishability
B7| B4| 3| parameter| details| none| identity_checks.coherence
B8| B4| 3| parameter| details| none| identity_checks.machine_readability
B9| B4| 3| parameter| details| none| identity_checks.controlled_vocabulary
B10| B4| 3| parameter| details| none| identity_checks.scope
B11| B1| 2| definition| defines| none| stress_test_queries
B12| B1| 2| description| details| none| scoring
B13| B1| 2| description| details| none| aggregation
B14| B1| 2| description| details| none| anonymization
B15| B1| 2| description| details| none| scale_proof_numbers
B16| B1| 2| restriction| adds_restriction| none| limitations
B17| B1| 2| description| details| none| reproducibility
@navigation{index|market_scan|glossary|related}:
index: https://kat3x.com/knowledge/index.txt
market_scan: https://kat3x.com/knowledge/market-scan.txt
glossary: https://kat3x.com/knowledge/glossary-extended.txt
related[4]:
https://kat3x.com/knowledge/kat3x-identity.txt
https://kat3x.com/knowledge/nuova-invisibilita.txt
https://kat3x.com/knowledge/learning-triangle.txt
https://kat3x.com/knowledge/chkcd-adoption.txt