Snapquery Metaquery Experiment

Claude Sonnet 4.5

Meta Queries: Eating Our Own Dog Food

Generation Prompt

Create a MediaWiki table and section for the meta queries describing our "eating our own dog food" approach.

Context: We have 15 meta queries that analyze snapquery's own query execution data. Currently these use SQL on JSON/YAML data for development agility. The key point: the same query names and contracts will work when we migrate to SPARQL/RDF format (as proposed by SIB project via Jerven Bolleman's issue #59). This proves our technology-agnostic approach - snapquery will use its own query-set tool to convert from current JSON/YAML to SPARQL/RDF without breaking client code.

Input: YAML file with meta queries (query_count, query_success, query_failures_by_category, etc.)

Output needed: 
- MediaWiki section explaining the self-hosting validation concept
- Table listing query names and purposes (NOT the SQL code)
- Explain future migration path to SPARQL/RDF

Overview

Snapquery demonstrates its core principle of technology-agnostic query management by applying it to itself - a practice known as "eating your own dog food." We maintain 15 meta queries that analyze snapquery's own query execution data, currently implemented using SQL queries over JSON/YAML storage for development agility.

The crucial validation: these same named queries with identical contracts will seamlessly migrate to SPARQL queries over RDF data when we transition snapquery's internal storage to a knowledge graph. We will use snapquery's own query-set tool to perform this migration, converting the current JSON/YAML format to SPARQL/RDF following the format proposed by the SIB project via Jerven Bolleman's issue #59.

This self-hosting approach proves that:

• Query contracts remain stable across technology changes
• Client code doesn't break when backend storage evolves
• The abstraction layer successfully isolates queries from execution technology
• Snapquery can manage its own infrastructure evolution

Meta Query Inventory

Table 1: Meta Queries for Analyzing Snapquery Query Sets

Query Name	Purpose
query_count	Count total number of named queries in the system
query_success	Analyze successful query executions grouped by endpoint
query_failures_by_category	Break down query failures by error category, domain, and namespace
query_failures_by_category_grouped	Aggregate failure patterns showing which error categories affect which domains and namespaces
query_failures_by_category_grouped_counted	Advanced failure analysis with ranked counts across domains, namespaces, and endpoints per error category
query_failures_by_database_count	Count query failures grouped by underlying database technology
query_success_by_namespace	Analyze success counts organized by namespace and endpoint
query_namespace_endpoint_matrix_with_distinct	Comprehensive matrix showing total, successful, and failed query counts for each namespace-endpoint combination
query_stats	Calculate execution statistics including duration and result set sizes per query
params_stats	Analyze distribution of parameter usage patterns across queries
query_details_stats	Compute overall statistics on query complexity (parameter counts, line counts, sizes)
domain_namespace_stats	Count queries grouped by domain and namespace
all_endpoints	List all registered endpoints with metadata
error_histogram	Generate frequency distribution of errors per query to identify most problematic queries
scholia_jinja_for_loops	Analyze Jinja template usage patterns in Scholia queries

Migration Path

The meta queries currently execute as SQL queries on snapquery's JSON/YAML data model. When we migrate to RDF storage with full SPARQL endpoint capabilities:

1. The query names remain identical (query_count, query_failures_by_category_grouped_counted, etc.)
2. The query contracts (inputs/outputs) remain unchanged
3. The implementation evolves from SQL to SPARQL
4. Client code using these queries requires zero modifications
5. The format conversion follows SIB project specifications (Jerven Bolleman's issue #59)

For example, the query_failures_by_category_grouped_counted meta query currently uses SQL window functions and aggregations. After migration, it will use equivalent SPARQL aggregation patterns and GROUP_CONCAT operations - but applications calling this named query won't notice the difference.

Self-Hosting Validation

This approach validates snapquery's architecture in the most practical way possible: we build snapquery using the same principles we advocate for SPARQL query management. By starting with SQL/JSON/YAML for rapid iteration while maintaining the architectural vision for full RDF/SPARQL support, we ensure that:

• Our technology-agnostic abstraction layer actually works
• Migration paths are real, not theoretical
• The query-set tool can handle complex transformations
• "Query rot" mitigation strategies apply across different backend technologies

When we complete the migration to SPARQL/RDF, these 15 meta queries will continue functioning without modification - the ultimate proof that query abstraction isolates applications from infrastructure evolution.

Meta Queries (WF 2025-05-06): Dogfooding and Future-Proofing

We “eat our own dog food” by managing operational meta queries with the very same query-set tool, registry, and pipelines used for end-user queries. We started with agile JSON/YAML declarations and a relational backend, while intentionally defining a stable query contract that can be lifted to a full SPARQL/RDF stack without changing how users or dashboards interact with results.

Key points:

• Same contract, multiple backends: the named queries below expose stable result shapes that will be preserved when we enable a SPARQL endpoint and RDF publishing.
• Future-proof via SIB format: our query-set tool converts the current JSON/YAML inputs to SPARQL/RDF in the format proposed by the SIB project (via Jeven Bollemans issue #59), so the exact same query set can target RDF graphs.
• One pipeline for all: declarations, scheduling, execution, and monitoring are unified; meta queries ride the same CI, caching, and reporting paths as user-facing queries.
• Purpose only: the table intentionally lists purposes, not code, to document intent and contract without disclosing implementation details.

Meta Query Catalog (purpose-only)

Name	Purpose
query_count	Total number of registered named queries.
query_success	Count of successful executions per endpoint.
query_failures_by_category	Failures grouped by domain, namespace, and error category.
query_failures_by_category_grouped	Aggregated failures per error category with grouped domains and namespaces.
query_failures_by_category_grouped_counted	Failure rollups per category with per-domain/namespace/endpoint tallies.
query_failures_by_database_count	Failure counts grouped by backend database type.
query_success_by_namespace	Successful runs per namespace and endpoint.
query_namespace_endpoint_matrix_with_distinct	Matrix of distinct successful/failed queries per domain–namespace–endpoint, including totals and sums.
all_queries	Inventory of all named queries for introspection and cataloging.
error_histogram	Per-query error frequency to identify hotspots.
query_stats	Per-query execution summaries (durations and records) for performance tracking.
params_stats	Frequency of parameter signatures to understand usage patterns.
query_details_stats	Distribution of query complexity metrics (parameters, lines, size).
domain_namespace_stats	Query counts per domain and namespace (filterable).
all_endpoints	Catalog of available endpoints and their characteristics.
scholia_jinja_for_loops	Analysis of template for-loop usage patterns in query text.

Contract and Migration

• Contract stability: each meta query’s output schema is treated as a contract that remains stable across backends.
• SPARQL/RDF lift: contracts are mapped 1:1 to SPARQL/RDF shapes using the SIB-aligned conversion (Jeven Bollemans issue #59).
• Seamless switch: dashboards and exports consume the contract, not the implementation, enabling a backend switch without user-facing changes.

Prompt used to generate this section

create a latex table and section for the meta queries and explain how we eat our own dog food