Difference between revisions of "Snapquery/LLMCommits"
| (2 intermediate revisions by the same user not shown) | |||
| Line 7: | Line 7: | ||
Mostly via parallel OpenRouter usage and mix and match of proposed modifications. | Mostly via parallel OpenRouter usage and mix and match of proposed modifications. | ||
We have used https://gitingest.com/ to partly try to provide the whole repository as a background for prompts and try general prompts such | We have used https://gitingest.com/ to partly try to provide the whole repository as a background for prompts and try general prompts such | ||
| − | as "Implement https://github.com/WolfgangFahl/pyLoDStorage/issues/151" ''simplify endpoints.yaml by referencing prefix sets in prefixes.yaml'' and cut and paste of the issue. | + | as "Implement https://github.com/WolfgangFahl/pyLoDStorage/issues/151" ''simplify endpoints.yaml by referencing prefix sets in prefixes.yaml'' and cut and paste of the issue github markup. |
| − | The results have only been partly useable but costly in terms of the number of prompt repetitions. The total usage cost was less than $35 during the whole refactoring period. | + | The results have only been partly useable but costly in terms of the number of prompt repetitions needed to get a useful result. The total LLM usage cost was less than $35 during the whole refactoring period - the fact that Grok 4 Fast Free was available has probably cut the cost by halfe. |
| − | There were cases where prompts had to be refined more than a dozen times and frustration kicked in how difficult it was to avoid the larger models to not try to produce too much code. | + | There were cases where prompts had to be refined more than a dozen times and frustration kicked in how difficult it was to avoid the larger models to not to try to produce too much code and go off the rails while following their trained statistical parrot trails. |
It was hard to contain damage by hallucinations that kept leaking in and could only be detected by the teste cases in the CI. | It was hard to contain damage by hallucinations that kept leaking in and could only be detected by the teste cases in the CI. | ||
| Line 36: | Line 36: | ||
{{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=fixes #65|name=Wolfgang Fahl|date=2025-12-02 14:43:18 +0100|hash=77d73d2|storemode=subobject|viewmode=line}} | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=fixes #65|name=Wolfgang Fahl|date=2025-12-02 14:43:18 +0100|hash=77d73d2|storemode=subobject|viewmode=line}} | ||
{{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=adds SIB sparql examples code|name=Wolfgang Fahl|date=2025-12-02 14:44:40 +0100|hash=db9cdef|storemode=subobject|viewmode=line}} | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=adds SIB sparql examples code|name=Wolfgang Fahl|date=2025-12-02 14:44:40 +0100|hash=db9cdef|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=adds triply endpoint|name=Wolfgang Fahl|date=2025-12-03 12:14:45 +0100|hash=1b08b6f|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=fixes #66|name=Wolfgang Fahl|date=2025-12-03 13:28:41 +0100|hash=67ec469|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=fixes wrong isodate in comment|name=Wolfgang Fahl|date=2025-12-03 13:29:23 +0100|hash=6110a30|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=adds action stats|name=Wolfgang Fahl|date=2025-12-03 13:31:03 +0100|hash=e31b916|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=adds zenodo DOI|name=Wolfgang Fahl|date=2025-12-03 13:42:02 +0100|hash=0c2393d|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=improves test_queries|name=Wolfgang Fahl|date=2025-12-03 13:49:01 +0100|hash=a9ac795|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=adds CITATION.cff|name=Wolfgang Fahl|date=2025-12-03 14:56:27 +0100|hash=c096d9c|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=Merge branch 'main' of https://github.com/WolfgangFahl/snapquery|name=Wolfgang Fahl|date=2025-12-03 14:56:36 +0100|hash=b82a0da|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=improves README|name=Wolfgang Fahl|date=2025-12-03 15:59:55 +0100|hash=ede1e31|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=fixes #67|name=Wolfgang Fahl|date=2025-12-04 07:06:17 +0100|hash=99cb504|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=fixes #68|name=Wolfgang Fahl|date=2025-12-04 08:51:32 +0100|hash=e78efac|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=fixes #69|name=Wolfgang Fahl|date=2025-12-04 09:28:26 +0100|hash=fc88d1a|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=added sib-examples command to QuerySetCmd|name=Tim Holzheim|date=2025-12-04 14:06:07 +0100|hash=ccf5e5b|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=snapquery|subject=added progress bar to sib fetcher|name=Tim Holzheim|date=2025-12-04 14:51:10 +0100|hash=303cd3d|storemode=subobject|viewmode=line}} | ||
| + | |||
== pyLoDStorage == | == pyLoDStorage == | ||
{{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=fixes #150|name=Wolfgang Fahl|date=2025-11-22 11:20:07 +0100|hash=b3e9d2e|storemode=subobject|viewmode=line}} | {{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=fixes #150|name=Wolfgang Fahl|date=2025-11-22 11:20:07 +0100|hash=b3e9d2e|storemode=subobject|viewmode=line}} | ||
| Line 65: | Line 80: | ||
{{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=fixes useragent|name=Wolfgang Fahl|date=2025-12-02 09:59:34 +0100|hash=172ba31|storemode=subobject|viewmode=line}} | {{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=fixes useragent|name=Wolfgang Fahl|date=2025-12-02 09:59:34 +0100|hash=172ba31|storemode=subobject|viewmode=line}} | ||
{{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=fixes #153|name=Wolfgang Fahl|date=2025-12-02 10:23:37 +0100|hash=03812a7|storemode=subobject|viewmode=line}} | {{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=fixes #153|name=Wolfgang Fahl|date=2025-12-02 10:23:37 +0100|hash=03812a7|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=adds usage to README|name=Wolfgang Fahl|date=2025-12-03 16:53:46 +0100|hash=3aac9da|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=prepares release|name=Wolfgang Fahl|date=2025-12-03 16:55:43 +0100|hash=9f839bc|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=adds CITATION.cff and Zenodo DOI|name=Wolfgang Fahl|date=2025-12-03 17:23:19 +0100|hash=0e9f3d7|storemode=subobject|viewmode=line}} | ||
| + | {{commit|host=https://github.com|path=WolfgangFahl|project=pyLoDStorage|subject=works on https://phabricator.wikimedia.org/T329368#11431817|name=Wolfgang Fahl|date=2025-12-04 10:33:46 +0100|hash=caf5c95|storemode=subobject|viewmode=line}} | ||
[[Category:snapquery]][[Category:LLM]] | [[Category:snapquery]][[Category:LLM]] | ||
Latest revision as of 05:06, 5 December 2025
AI supported refactoring release 0.1.0 to 0.2.1
In the refactoring effort the following LLMs have been involved:
- ChatGPT-4, ChatGPT-5
- Claude Sonnet 4.5
- Gemini3 Pro Preview
- Grok 4 Fast Free
Mostly via parallel OpenRouter usage and mix and match of proposed modifications. We have used https://gitingest.com/ to partly try to provide the whole repository as a background for prompts and try general prompts such as "Implement https://github.com/WolfgangFahl/pyLoDStorage/issues/151" simplify endpoints.yaml by referencing prefix sets in prefixes.yaml and cut and paste of the issue github markup. The results have only been partly useable but costly in terms of the number of prompt repetitions needed to get a useful result. The total LLM usage cost was less than $35 during the whole refactoring period - the fact that Grok 4 Fast Free was available has probably cut the cost by halfe. There were cases where prompts had to be refined more than a dozen times and frustration kicked in how difficult it was to avoid the larger models to not to try to produce too much code and go off the rails while following their trained statistical parrot trails. It was hard to contain damage by hallucinations that kept leaking in and could only be detected by the teste cases in the CI.
The following commits might contain LLM generated code:
snapquery
snapquery/WolfgangFahl:upgrades (Wolfgang Fahl/2025-12-01 16:09:35 +0100)
snapquery/WolfgangFahl:adds missing pandas dependency (Wolfgang Fahl/2025-12-01 16:47:48 +0100)
snapquery/WolfgangFahl:reformats (Wolfgang Fahl/2025-12-01 16:56:16 +0100)
snapquery/WolfgangFahl:fixes endpoints.yaml syntax (Wolfgang Fahl/2025-12-01 17:16:53 +0100)
snapquery/WolfgangFahl:improves endpoints test (Wolfgang Fahl/2025-12-01 17:27:28 +0100)
snapquery/WolfgangFahl:refactors to use lod_storage endpoint manager (Wolfgang Fahl/2025-12-01 19:23:47 +0100)
snapquery/WolfgangFahl:fixes dependency (Wolfgang Fahl/2025-12-01 19:53:57 +0100)
snapquery/WolfgangFahl:remove circular import (Wolfgang Fahl/2025-12-01 19:56:05 +0100)
snapquery/WolfgangFahl:upgrades to pyLoDstorage 0.18.2 (Wolfgang Fahl/2025-12-01 20:12:49 +0100)
snapquery/WolfgangFahl:upgrades to 0.18.3 pyLodStorage (Wolfgang Fahl/2025-12-01 21:07:20 +0100)
snapquery/WolfgangFahl:fixes Basetest reference (Wolfgang Fahl/2025-12-02 06:42:29 +0100)
snapquery/WolfgangFahl:upgrades to pyLoDStorage 0.18.4 (Wolfgang Fahl/2025-12-02 06:53:03 +0100)
snapquery/WolfgangFahl:fixes dblp endpoint prefix sets (Wolfgang Fahl/2025-12-02 06:56:00 +0100)
snapquery/WolfgangFahl:refactors query extractor (Wolfgang Fahl/2025-12-02 07:59:36 +0100)
snapquery/WolfgangFahl:fixes more tests (Wolfgang Fahl/2025-12-02 08:15:55 +0100)
snapquery/WolfgangFahl:tries fixing whitespace issue in SPARQL analyzer (Wolfgang Fahl/2025-12-02 08:21:38 +0100)
snapquery/WolfgangFahl:rearranges badges in README (Wolfgang Fahl/2025-12-02 09:24:19 +0100)
snapquery/WolfgangFahl:prepares release (Wolfgang Fahl/2025-12-02 10:54:41 +0100)
snapquery/WolfgangFahl:refactors github access to separate module (Wolfgang Fahl/2025-12-02 11:52:12 +0100)
snapquery/WolfgangFahl:skip sib access (Wolfgang Fahl/2025-12-02 11:56:48 +0100)
snapquery/WolfgangFahl:fixes #65 (Wolfgang Fahl/2025-12-02 14:43:18 +0100)
snapquery/WolfgangFahl:adds SIB sparql examples code (Wolfgang Fahl/2025-12-02 14:44:40 +0100)
snapquery/WolfgangFahl:adds triply endpoint (Wolfgang Fahl/2025-12-03 12:14:45 +0100)
snapquery/WolfgangFahl:fixes #66 (Wolfgang Fahl/2025-12-03 13:28:41 +0100)
snapquery/WolfgangFahl:fixes wrong isodate in comment (Wolfgang Fahl/2025-12-03 13:29:23 +0100)
snapquery/WolfgangFahl:adds action stats (Wolfgang Fahl/2025-12-03 13:31:03 +0100)
snapquery/WolfgangFahl:adds zenodo DOI (Wolfgang Fahl/2025-12-03 13:42:02 +0100)
snapquery/WolfgangFahl:improves test_queries (Wolfgang Fahl/2025-12-03 13:49:01 +0100)
snapquery/WolfgangFahl:adds CITATION.cff (Wolfgang Fahl/2025-12-03 14:56:27 +0100)
snapquery/WolfgangFahl:Merge branch 'main' of https://github.com/WolfgangFahl/snapquery (Wolfgang Fahl/2025-12-03 14:56:36 +0100)
snapquery/WolfgangFahl:improves README (Wolfgang Fahl/2025-12-03 15:59:55 +0100)
snapquery/WolfgangFahl:fixes #67 (Wolfgang Fahl/2025-12-04 07:06:17 +0100)
snapquery/WolfgangFahl:fixes #68 (Wolfgang Fahl/2025-12-04 08:51:32 +0100)
snapquery/WolfgangFahl:fixes #69 (Wolfgang Fahl/2025-12-04 09:28:26 +0100)
snapquery/WolfgangFahl:added sib-examples command to QuerySetCmd (Tim Holzheim/2025-12-04 14:06:07 +0100)
snapquery/WolfgangFahl:added progress bar to sib fetcher (Tim Holzheim/2025-12-04 14:51:10 +0100)
pyLoDStorage
pyLoDStorage/WolfgangFahl:fixes #150 (Wolfgang Fahl/2025-11-22 11:20:07 +0100)
pyLoDStorage/WolfgangFahl:fixes description declaration (Wolfgang Fahl/2025-11-22 12:13:51 +0100)
pyLoDStorage/WolfgangFahl:towards #151 (Wolfgang Fahl/2025-11-22 14:12:36 +0100)
pyLoDStorage/WolfgangFahl:towards #151 (Wolfgang Fahl/2025-11-22 16:05:18 +0100)
pyLoDStorage/WolfgangFahl:adds new testcase and improves (Wolfgang Fahl/2025-11-22 23:01:50 +0100)
pyLoDStorage/WolfgangFahl:refactors YamlPath to separate module (Wolfgang Fahl/2025-11-23 11:11:05 +0100)
pyLoDStorage/WolfgangFahl:improves prefix sets (Wolfgang Fahl/2025-11-23 12:57:06 +0100)
pyLoDStorage/WolfgangFahl:fixes prefix configuration test (Wolfgang Fahl/2025-11-23 13:25:43 +0100)
pyLoDStorage/WolfgangFahl:migrates ActionStats to separate module (Wolfgang Fahl/2025-11-23 13:28:36 +0100)
pyLoDStorage/WolfgangFahl:refactors (Wolfgang Fahl/2025-12-01 09:40:28 +0100)
pyLoDStorage/WolfgangFahl:fixes mtriples for qlever endpoints (Wolfgang Fahl/2025-12-01 09:47:39 +0100)
pyLoDStorage/WolfgangFahl:adds getInstance for PrefixConfigs (Wolfgang Fahl/2025-12-01 10:06:59 +0100)
pyLoDStorage/WolfgangFahl:tries fixing CI (Wolfgang Fahl/2025-12-01 10:20:14 +0100)
pyLoDStorage/WolfgangFahl:fixes of_yaml calls (Wolfgang Fahl/2025-12-01 10:31:03 +0100)
pyLoDStorage/WolfgangFahl:adds Exception handler and makes QueryManager constructor more robust (Wolfgang Fahl/2025-12-01 10:44:38 +0100)
pyLoDStorage/WolfgangFahl:fixes #151 (Wolfgang Fahl/2025-12-01 11:17:07 +0100)
pyLoDStorage/WolfgangFahl:prepares release (Wolfgang Fahl/2025-12-01 15:10:53 +0100)
pyLoDStorage/WolfgangFahl:improves README (Wolfgang Fahl/2025-12-01 15:14:31 +0100)
pyLoDStorage/WolfgangFahl:fixes debug json dump (Wolfgang Fahl/2025-12-01 15:21:42 +0100)
pyLoDStorage/WolfgangFahl:less debug output in CI (Wolfgang Fahl/2025-12-01 15:23:53 +0100)
pyLoDStorage/WolfgangFahl:integrateds snapquery compatible EndpointManager features (Wolfgang Fahl/2025-12-01 19:10:55 +0100)
pyLoDStorage/WolfgangFahl:adds get_samples compatibility (Wolfgang Fahl/2025-12-01 20:04:59 +0100)
pyLoDStorage/WolfgangFahl:fixes get_samples (Wolfgang Fahl/2025-12-01 21:00:14 +0100)
pyLoDStorage/WolfgangFahl:adds litre and dacite prefixes (Wolfgang Fahl/2025-12-02 06:40:29 +0100)
pyLoDStorage/WolfgangFahl:prepares release (Wolfgang Fahl/2025-12-02 06:42:21 +0100)
pyLoDStorage/WolfgangFahl:tries fixing rate limiting and user agent (Wolfgang Fahl/2025-12-02 09:52:36 +0100)
pyLoDStorage/WolfgangFahl:fixes useragent (Wolfgang Fahl/2025-12-02 09:59:34 +0100)
pyLoDStorage/WolfgangFahl:fixes #153 (Wolfgang Fahl/2025-12-02 10:23:37 +0100)
pyLoDStorage/WolfgangFahl:adds usage to README (Wolfgang Fahl/2025-12-03 16:53:46 +0100)
pyLoDStorage/WolfgangFahl:prepares release (Wolfgang Fahl/2025-12-03 16:55:43 +0100)
pyLoDStorage/WolfgangFahl:adds CITATION.cff and Zenodo DOI (Wolfgang Fahl/2025-12-03 17:23:19 +0100)
pyLoDStorage/WolfgangFahl:works on https://phabricator.wikimedia.org/T329368#11431817 (Wolfgang Fahl/2025-12-04 10:33:46 +0100)