top of page
Search

My Weekend Nerd Challenge: Build a Proof-of-Concept App Using Only ChatGPT

Writer's picture: RobbieRobbie

Some of you will have worked out by now that I have a nerdy streak that’s hard to hide. This post is about my latest weekend nerd challenge.


To keep my tech skills fresh, I often do small self challenges - a few months ago I built Mack, a computer vision AI model with JavaScript front end that identifies, geotags, and logs potholes in the road, using AI to craft reports to the local council.


I’ve built an example proactive diabetes diagnostic tool using Dexcom CGM data. I’ve also built a Python based accidental OSINT intelligence gathering tool that I proved in the wild and managed to track the movements of a very high profile organisations board of directors (and this is still under responsible disclosure embargo 🤣)


This weekend, I challenged myself to write a proof of concept app snippet. It had to be written entirely by prompting ChatGPT. At no point was I allowed to manually edit the JavaScript where I saw errors - natural language prompting only!


The entire job had to be done on Safari, the iPhone web browser only.


(This last requirement quickly got adjusted when I needed to get a dataset from an FTP location, and it came down as a .gz - I did allow myself to use FTPManager and iZip to open these!).


And I had to complete the task in however long it took my HP Spectre to hit 0% battery from whatever it was charged to, and shutdown (that turned out to be 4hrs 30mins).


It also had to run client side (so no PHP!) and I wanted to be able to show live code (so no authenticated APIs), without risking uncontrolled API cost accrual.


Finding a Problem to Solve


I needed something interesting but achievable in the time limit, and I wanted it to be something I have no real technical authority in. I thought about stuff that's annoyed me over the years.


When I lived in Croydon, I had access to a lot of train stations - Purley Oaks, Sanderstead, South Croydon, East Croydon, West Croydon, Kenley, and Coulsdon South, all running services to Victoria.


Trouble was, being equidistant from South Croydon, Sanderstead, and Purley Oaks meant I could take any of the three. East Croydon was only a 10-minute drive.


Throughout the day, different trains would stop at different stations - some at Sanderstead but not Purley, some only at East Croydon, some rare ones at both. Planning a journey meant checking multiple stations individually. I’d be standing at one station only to watch a train to the other station sail past me.


Now, Google Maps won’t factor in travel time convenience, and on Trainline, you have to choose your start location and end location, with no ability to add multiples / general geographic area, so you have to judge which is likely to be the best.


I needed a many-to-many data broker, something that:


  1. Lets you select multiple start locations (stations, postcodes, or GPS location).

  2. Finds all stations within a chosen X miles radius.

  3. Estimates travel time to those stations (using crow-flies miles divided by 45mph).

  4. Generates Trainline links (or Google Maps links) for each start-end combination.


A proper version would use the Google Directions API, dedupe results, and present a single table. But I wasn’t allowing authenticated APIs, so this was very much proof-of-concept mode.


Its value would be to show me the optimal route from my location to an end-point, via the train system, where my start point could vary based on the timetable, and distance to alternate stations, prepending the train travel time with the car journey time to allow me to select my favoured route.


What Worked (and What Didn’t)


It mostly worked - until I hit Trainline’s station ID format, which broke my original plan.


I wanted the app to trigger Trainline searches automatically, but I couldn’t figure out their from/to format. URLs contained station identifiers like:


urn:trainline:generic:loc:VIC5426gb

The UK Rail Delivery Group dataset only listed station codes in three-letter format (VIC for Victoria), but here was this unexplained 5426 number stuck onto the end.


I spent 20 minutes trying to get ChatGPT to work it out, but it had no idea what 5426 represented. It suggested contacting Trainline. (Thanks, ChatGPT.)


This is where where human intuition kicks in.


Something about the four-digit code felt off. If it were arbitrary, I’d expect it to change between searches. It didn’t. That meant it could be proprietary, but it also could be a reference to something else. If it's proprietary, I figured I could scrape the data with 1500 easily scriptable calls... not good practice though.


Naturally, my first instinct was to Google the question. No luck.


I tried searching for “UK station ID datasets” and landed on a Trainline (handy!) GitHub repo with an EU train stations dataset. It had UIC (Unique Identifier Code), coordinates, parent stations, and - most importantly - numeric identifiers.


When I cross-referenced the dataset against the URL-based references I had, I discovered the four-digit 5426 was actually the station ID, buried inside the broader UIC field, which is actually a composite data field. I looked for the UIC8_SNCF specification, noted the first 2 characters are country code (70 for UK), the next 4 are for Unique Station Identifier (5426 for London Victoria), and the last 1 / 2 digits are a checksum.


Presumably because Trainline is global rather than just UK based, so they were looking for full unique primary keys.


I realised I’d spent way too much time chasing a mystery code instead of actually integrating it into the app. I'd taken a wrong turn at the early point at the insistence of ChatGPT that its "just a random number". Nooooo!


My laptop battery was at 4%, and it decided this exact moment was time to call it a day. I had the answer, at exactly the wrong moment!


In my head, I heard Noel Fielding, “Step away from your bakes”.  No Hollywood handshake for me.


Humans vs. AI: Heuristic Thinking vs. Token Windows


The code itself is ok for proof of concept, and I got further than I thought I would, given the constraints, but I wish I’d worked out the Trainline schema sooner - I couldn’t hit on a good prompt for ChatGPT to work it out, and it just kept giving up and telling me to contact Trainline.


There’s obviously a very human heuristical analysis going on when my brain is going “no it can’t be right, this doesn’t feel right” that the LLMs are up to yet; instinct vs reasoning? It my non-linear ADHD way of thinking, and gut feel, my USP in the inevitable Humans Vs AI battle? Is it the fact that my brain can openly disagree with itself, create subconsciously Kahneman-slow-brain style, switching between fast and slow to long list / shortlist? Or is it just that I have a longer token window?


Could it be that I’m trying to achieve an outcome from my own mind, which formed because of a theory I’ve created from my own experience, in a challenge where I’ve set myself unreasonable parameters - and so I’m ahead of the LLM, because it’s not a mind reader, and all it’s getting from me is carefully crafted prompts at a 60-80 words per minute baud rate, whereas I’m hearing my own thoughts in real time, at, presumably, a much faster rate.


(Aside: Copilot informs me that the brain processes,, subconsciously and consciously, around a billion bits per second, or 119 Terabytes per second (let’s not have a conversation about transit vs storage right now - if the global internet throughput in 2022 was 150gbps, and a brain throughput is 119 tbytes/ 952 tbits per second, then your brain is processing more than 6x the total throughput of the internet. Tbh, I thought my mind sounded noisey! 🤣)



Takeaways: AI Isn’t Replacing Developers Yet


I was trying to discover:


  1. how annoying it’d be (answer… it’s really annoying, ChatGPT just forgets stuff, changes code you’ve agreed, etc),

  2. how close ChatGPT is to Nvidia’s Jensen Huang’s “you won’t need to program, you just need to prompt” vision (answer…. It’s a long way off - but easily imaginable with the tools currently available)

  3. what would happen when we hit a problem where my natural instinct is to stare at the code and speculate on causes (answer… GPT just couldn’t / wouldn’t be drawn on the station code problem, but once I solved it, it carried on as if it knew the answer all along).


ChatGPT wasn't able to deduce the UIC format on its own. But the moment I told it the answer, it confidently continued as if it had known it all along.


I’d been struck by the recent conversations about Agentic AI, around how web services should be built with a robots.txt style document to tell the agents how to interact with the web services…


I’m not sure I can see how that works where most of these APIs are monetised by use, where the agent will need to have API usage tokens, and where the early version of the code (before I told it to stop trying authenticated APIs) was trying to call Google Directions for every journey type, and 6 times per journey.


Considering it’s about $0.01 per call to Google Directions, a 10 million API call budget of $100k could serve 5-10m customers / requests in well crafted code, but in the ChatGPT generated code, making 6 calls, across 7 stations, that’s 42 calls per call, reducing my $100k API budget value to just $2,380 through inefficiency. How do I catch that in prompting? Surely I still need to be able to read, and understand the code, to catch issues, or be happy with deploying high risk code.


The "thing" itself...


Here's the horror show that I have created, and usage:


  1. Set one or more start stations

  2. Set one of more end stations

  3. Set a start radius in miles

  4. If you chose multiple start locations, you can choose a preferred one

  5. Set a journey start time

  6. Hit "view routes" to see a list of queries it'd run against Google

  7. Hit "view all routes" to see roughly what the data return would be (note: this'll open each route in a new tab!)



Things I wish I'd got to in time:


  1. The CSV dataset from Trainline EU, and built the script around that rather than the Rail Delivery Group's JSON

  2. Changing the "start station" so it could take a postcode - I'd probably have used OpenStreetMap to get geographical coordinates for that, and could've used the local GPS location, or a postcode as the start point, rather than hard coding a station.

  3. Working out that Trainline used the UIC station code as part of its station identifier... it's obvious now, but I started the "project" without any knowledge whatsoever around whether this data existed, what the data meant, or how to use it.

  4. If I'd been able to spend some more time on it, I might've included a text box to drop an API key into; so that I could release client side code with a Bring Your Own API Key idea. But then, how many people have got API keys for Google Directions? 😄

  5. Fixing the journey start time issue; it doesn't pass to Google. The final cut removed the estimated travel time, and represents it only in miles.

  6. Looking at the code to work out why Preferred Start time populates on some versions of this code after View Routes is pressed (Rather than using a field update event trigger)

  7. Aesthetics and accessibility.


Aside from that I think we can agree, that based on the evidence, a fun day was had by all.


Links:

  • Trainline EU Stations Data: GitHub

  • UK Rail Delivery Group Dataset: Open Data

0 views0 comments

Recent Posts

See All

Comentários

Avaliado com 0 de 5 estrelas.
Ainda sem avaliações

Adicione uma avaliação

© 2023 Random Creation House Ltd

bottom of page