My Problem

At the moment I have a bit of a contrived setup for importing photos from my camera. I have a Raspberry Pi permanently setup to listen for the insertion of my camera’s memory card. The Pi automatically copies the images (and any videos) to a timestamped folder on my network attached storage (NAS) under a directory for the current year. I’ve found this to be a convenient and neat way to import and organise stuff. I later rename the timestamped folder to something more appropriate like “Father’s Day.”

Since getting a new camera, I have started using Adobe Lightroom, and it has proven to be worth the investment. My workflow now involves manually creating a corresponding album in Lightroom, importing RAW photos from my NAS, editing them in Lightroom, and then saving both small and large JPEG versions back to my server as an archive.

Why not just store everything in Lightroom? Although it can store videos, the space would be used up very quickly, and upgrading to a higher storage tier would be expensive. My impression is that Lightroom storage is not designed for long-term archiving. It feels more like a scratch space, intended for editing your current work with just enough room to do so comfortably. Once finished, albums should be archived somewhere more permanent. For this reason, I find it necessary to bring the edited photos back onto my NAS. With the images on my NAS, I also gain the ability to conveniently view the photos and videos on our smart TV’s.

As I mentioned, the process can be a bit cumbersome. If I export images from Lightroom and later decide to make further edits, perhaps after learning something new or spotting ways to improve them, I have to export the images again and replace the versions stored on my NAS. Keeping everything in sync is a constant hassle.

The initial import from my NAS into Lightroom can also be quite slow. Each RAW photo taken by my camera is around 44MB, and transferring hundreds of them over WiFi can be a frustrating process.

How the Lightroom API Could Help Me

By using the Lightroom API, my hope would be that I could produce an application that would allow:

1. My Raspberry Pi to create and fill albums in Lightroom directly when I insert my camera’s memory card.

2. Synchronisation of the small and large JPEG image versions with my server with changes detected by events.

This would mean no waiting for images to import to Lightroom, and no worry over the local images being in sync with edits I make on Lightroom.

Using the Lightroom API

The first step in using the Lightroom API is to produce an integration to get an API key for the Lightroom API. To do this I followed the steps outlined here:

1. Logged into the Adobe I/O Console using my usual credentials.

2. Created a project.

3. Selected “Add API”

4. Added the “Lightroom Services API”

5. Selected “OAuth Single-Page App”

With the API key (Client ID), I could then use Postman to follow the OAuth authentication flow, generate access tokens, setup each endpoint using the API reference, and test if I could use them to:

• Upload an image ✅

• Access a high-resolution version of an image ✅

• Access a low-resolution version of an image ✅

• Create an album ❌

• Add images to album ✅

I’ve made the workspace I was using public just in case it’s helpful to anyone trying to get to grips with the API. The workspace may be accessed using the button below.

Unfortunately, these tests confirmed that it was not possible to create an album using the Lightroom API in the way that I had wanted. Adobe has implemented this feature in such a way that albums created using the Lightroom API will not show in the Lightroom application. This is confirmed by a community forum post where the Adobe employee replies, “albums are specific to the partner who has created the album. It is only meant to seen on the partner and not on Lightroom applications.“ This is quite frustrating!

I think I would be able to tolerate having to manually create albums in Lightroom, it would still be a big win being able to upload images automatically. Next (and in another blog) I would like to explore events to confirm if it would be possible to trigger some sort of sync when a photo (asset) is edited. Only if this is possible would I pursue this project further.

What is Nomad?

Nomad is a flexible scheduler and workload orchestrator that enables you to deploy and manage any application across on-premise and cloud infrastructure at scale.

That’s the official definition. To put it more simply, Nomad is like a logistics manager for software. As a developer, you define what your application needs and how it should run, and Nomad (the logistics expert) decides how and where to deploy it across your available infrastructure.

Advertised Features

Efficient resource usage
Nomad makes smart use of your available infrastructure by packing tasks together on machines in a way that avoids waste. This approach, known as bin packing, helps ensure resources like CPU and memory are fully used without overloading any one system.

Self-healing
Nomad keeps an eye on all running tasks. If something crashes or stops responding, it automatically restarts or reschedules it elsewhere, helping to keep your services available and resilient.

Zero downtime deployments
Nomad supports safe deployment strategies that prevent user disruption. Rolling updates replace old versions with new ones gradually. Blue/green deployments run the new version alongside the old one until it is ready to fully take over. Canary deployments roll out changes to a small portion of users first, allowing issues to be caught early before affecting everyone.

Supports many workload types
Nomad is highly flexible, able to run a wide range of applications. This includes container workloads like Docker, Java applications in JAR files, virtual machines via QEMU, or even raw scripts and system commands.

Cross-platform and portable
Nomad runs as a single lightweight binary and works across Linux, Windows, and macOS. You can use it to manage workloads running in your data centre, in the cloud, or at the edge – all from one system.

Simple, consistent job setup
You describe each application using a clear job file, where you define what the app is, how it should run, where it should run, and how it connects to other services. This means whether you're running a container, a VM, or a script, the process for setting it up in Nomad is always the same.

Key Terms

The following image was taken from an introductory Nomad tutorial, and I think gives a good overview of the core elements of the Nomad infrastructure.

Setup Terms

Agent
An agent is a Nomad process that runs either as a server or a client. It forms the core of any Nomad deployment.

Client
A Nomad client runs the actual tasks. It registers itself with the servers and waits for work to be assigned. Clients are often called nodes, especially when discussing the infrastructure.

Server
Nomad servers handle all job scheduling and manage the clients. They decide where tasks should run and monitor the overall health of the system.

Development agent
A development agent is a special configuration used for local testing or learning. It runs as both a server and a client on the same machine and does not save any data to disk. This means it always starts in a clean, predictable state, making it ideal for quick experiments or demos.

Operational Terms

Task
A task is the smallest unit of work in Nomad. It runs through a task driver such as Docker or Exec, which allows Nomad to support different types of workloads. Each task defines which driver it needs, along with its configuration, constraints, and required resources.

Group
A group is a collection of tasks that are run together on the same client. Tasks in a group share resources and are scheduled onto the same machine.

Job
A job is the main way to define and manage an application in Nomad. It includes everything Nomad needs to know to run one or more tasks, such as configurations, constraints, and deployment rules.

Job specification
Also called a ‘jobspec’, this is the complete definition of a job. It includes details such as the type of job, its tasks and resource needs, where it can run, and how it should behave. It’s written in a clear, structured format so Nomad can understand and act on it.

Allocation
An allocation is the result of Nomad placing a job on a specific client. It represents the link between a task group and the machine where it runs. When a job is launched, Nomad selects a suitable client and reserves resources on it for the job's tasks.

Why Use Nomad?

Having worked on MOONS instrument control software, I am acutely aware of the number of (to use Nomad jargon) ‘tasks’ that will be required of the system, and that they may be distributed across several instrument or detector control workstations. Immediately, the use-case for Nomad becomes clear. As a method to define, deploy, manage, and oversee these hundreds of tasks, it seems like a wise choice. The idea that Nomad can automatically handle task distribution, restart failed processes, and ensure smooth deployments across multiple machines makes a lot of sense, especially for something as complex as the ELT system. Given all these capabilities, Nomad feels like it could play a crucial role in ensuring everything runs smoothly across these workstations on future Very Large Telescope (VLT) and ELT instruments.

One concern I had early on was reliability. The server was running precariously on a Raspberry Pi: if the Pi rebooted, the server wouldn't start automatically. If the server crashed for any reason, it would stay down until someone manually restarted it. For the person I was designing this for, it was absolutely essential that the system "just worked." If it became unreliable, there was a real risk they would lose faith, and with it their willingness to engage with the system at all.

Today, I implemented a proper solution using systemd, the standard service manager for Linux systems. Systemd allowed me to define my Python server as a service that automatically starts at boot, monitors itself, and restarts if it crashes unexpectedly. This small change will hopefully improve the overall reliability of the system.

How to set it up

1. Create a systemd service file (example: /etc/systemd/system/yourservicename.service):

    [Unit]
    Description=My Python Application  
    After=network.target # Wait for network to load
   
    [Service]
    ExecStart=/path/to/venv/bin/python /path/to/your_app.py
    WorkingDirectory=/path/to/working/directory # ensures relative file paths work correctly.
    Restart=always # Always restart if the server crashes
    RestartSec=5  # Wait 5s before restarting the server 
    User=pi          
   
    [Install]
    WantedBy=multi-user.target # Start the service when we reach a fully booted system
   

2. Enable and start it:

    sudo systemctl daemon-reload
    sudo systemctl enable yourservicename.service
    sudo systemctl start yourservicename.service
   

3. Check its status:

    sudo systemctl status tv_control.service
   

Final thoughts

By setting up the Python server as a systemd service, I've taken an important step toward ensuring that the system remains dependable and frictionless for the person using it. They shouldn’t have to think about whether the server is running or worry about fixing anything if something goes wrong, it should just work, quietly and reliably in the background. This approach not only improves technical stability but also builds trust, making it far more likely that the system will become a comfortable and lasting part of their daily routine.

Continuing with my teacher’s metaphor, people who came to digital technology later in life could be called digital immigrants. Unlike digital natives, they didn’t grow up with the design patterns and workflows that underpin modern devices and interfaces. They might not be fluent in our digital language of menus, windows, gestures and forms. It makes sense that tasks like navigating a smart TV menu, installing an app, or signing into a streaming service could feel foreign.

My Digital Immigrant

There is one digital immigrant I spend time with often. Someone from the land of rotary phones, tube radios, and cutting-edge 1940s technology like black-and-white television. Unfortunately they have reached the “disheartened” or “uninterested” phase of digital engagement. To them, modern technology feels confusing and needlessly complex. Concepts that seem second nature to me, like the distinction between the TV remote and the set-top box (Sky television) remote, remain hard to internalise, even after repeated explanations. Opening an app on the television feels, to them, like trying to operate a super computer.

This individual has a favourite YouTube channel they enjoy watching, but they can only access it when someone is around to help. They need assistance to open the app, update it if necessary, search for the channel, open it, and choose a video they haven’t already seen. Just as I wouldn’t be able to follow instructions in Italian, they struggle with the unfamiliar language of digital technology. As a result, they often settle for whatever comes on when they turn on the TV - or nothing at all, especially if the set-top box doesn’t behave exactly as they expect.

With the television being this persons main source of entertainment, I wanted to try and think of a way to make their favourite YouTube channel more accessible to them.

The Plan

Find a way to, with the press of a single button:

1. Turn on the TV

2. Open YouTube

3. Automatically Choose the “watch as guest” option - they don’t have or want an account.

4. 

   Play a video from their favourite youtube channel that they haven’t seen before.

Controlling the TV Programatically

The Research

When I first started thinking about how to control the TV, I explored a few initial ideas, the first involving a microcontroller and IR blaster. The concept was simple: learn the remote codes and transmit a sequence of IR signals to turn the TV on, open YouTube, and perform other basic functions. But the more I considered it, the more fragile the solution seemed. For instance, if the TV was already on, sending an "on" command would actually turn it off. And if someone changed the order of apps on the home screen, the carefully timed sequence of button presses to launch YouTube would no longer work. The whole setup would rely on a brittle choreography of commands that could easily go out of sync. It would have been error prone and frustrating for the user.

Then I turned to the TV’s manual and was excited to find documentation about external control via a serial interface. This approach promised much more reliable functionality. You could send a definitive "power on" command, directly switch to a specific input source by name, or set an exact volume level; eliminating the guesswork and inconsistency of IR-based control. It was exactly the level of precision I was hoping for. Unfortunately, when I inspected the back of the TV, I realised there was no serial port available. I suppose this feature is only present on commercial models, and the manual was simply covering all possible variants. It was a bit of a letdown.

After more investigating I came across a Git Hub repository that contained a Python library that would let me connect to the TV and do all sorts of things. Crucially it would let me:

1. Connect to the television from a Python program (that will run on a Raspberry Pi)

2. Open YouTube by name, and even a specific YouTube video!

3. Programatically navigate and select the “watch as guest” option with the libraries facility to emulate basic up, down and ok button presses.

In addition it would let me:

1. Set the volume to a specific number.

2. Send notifications to the TV which I could use to tell the user what’s going on.

The only thing this library couldn’t do was turn the TV on - a very important step. However, I realised (and the documentation for this library points out) that you can use Wake on LAN and send a ‘magic’ packet to the television to wake it - you just need to enable this feature on the TV.

With this library I am now able to accomplish steps 1 to 3 of ‘the plan’.

The Implementation

Environment Variables

There are three runtime parameters that are needed in this program.

TV_ADDR="XXX.XXX.XXX.XXX" # IP address of the television (or hostname if you're fancy)
TV_MAC="FF:FF:FF:FF:FF:FF" # Hardware address of the network interface on the TV
WEBOS_KEY="11c4bc438d420b33cc2334323d77dddf" # Access key returned by registering with TV

I have chosen to store them in a .env file and use the load_dotenv() function to allow the values of the environment variables to be used by the program. This has to be done before any function that uses these parameters is used.

from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

Turning on the TV

The first component I needed was a function to turn the TV on using Wake on LAN. A quick Google revealed the wakeonlan Python package, which was easily installed with pip. All I have to do is send the magic packet to the TV’s hardware address to get it to turn on. In testing I found that sending it twice was always reliable.

from wakeonlan import send_magic_packet

def turn_on_tv():
    tv_mac = os.getenv('TV_MAC')

    # Turn on the TV through wake on LAN
    send_magic_packet(tv_mac)
    send_magic_packet(tv_mac)

Determining if the TV is on the Network

I then wanted a component to indicate if the TV was online. If its already on, I don’t have to turn it on. I also don’t want to attempt to connect to the TV before the it is on the network. This function simply pings the television, and if it responds within a second we can be pretty confident its online. If not, then it probably isn’t.

import os 

def tv_online():
    tv_addr = os.getenv('TV_ADDR')

    # Check if the TV is online by pinging it 
    # -c 1 (send one packet)
    # -t 1 (wait at most one second for response)
    response = os.system(f"ping -c 1 -t 1 {tv_addr}")
    if response == 0:
        print(f"{tv_addr} is online")
        return True
    else:
        print(f"{tv_addr} is offline")
        return False

Connecting to the Television

The next piece of functionality was establishing a connection to the television. For this, I used code largely borrowed from the Python library I mentioned earlier. The first time you attempt to connect, the tv_key should be an empty dictionary. Without a client key, the TV will display a pop-up asking you to confirm the connection. Once you approve it on-screen, the registration completes and the program will print out a unique client key.

This key can then be added to a .env file and the tv_key line in the script can be uncommented. From that point forward, the program can connect automatically without requiring manual confirmation each time.

In my implementation, I added a loop that retries the connection every five seconds, up to a maximum number of attempts. I noticed that even when the TV was online and responding to pings, the WebOS service sometimes wasn’t ready to accept connections immediately. This retry loop gives the TV time to fully boot its network services. If it still can’t connect after several tries, the script gives up; useful in cases where something is genuinely wrong.

Ideally, I’d like to improve this in the future by making the registration process fully automatic, including storing the client key persistently. But for now, this solution works well for my needs.

def connect_to_tv(max_tries=10):
    tv_key = {}
    #tv_key = {'client_key': os.getenv('WEBOS_KEY')}
    tv_addr = os.getenv('TV_ADDR')

    for i in range(max_tries):
        try:
            # Establish Connection With LG WebOS TV
            client = WebOSClient(tv_addr)
            client.connect()
            for status in client.register(tv_key, timeout=10):
                if status == WebOSClient.PROMPTED:
                    print("Please accept the connect on the TV!")
                elif status == WebOSClient.REGISTERED:
                    print("Registration successful!")
                    print(f"Store this as the environment variable WEBOS_KEY: {tv_key}")
                    return client
        except Exception as e:
            print(f"Connection attempt {i+1} failed: {e}")
            time.sleep(5)

Pulling it all Together

With all the key components in place, I built a main function to bring everything together and automate the experience from start to finish. Here's what it does:

• If the TV isn't already online, it powers it on and waits for it to become available on the network.

• It establishes a connection using a previously registered key.

• It sets the volume to a comfortable level (low enough to avoid surprises).

• It notifies the user that everything is working and YouTube will launch shortly.

• It fetches a new video from the user’s favourite channel (implementation covered in the next section).

• It launches YouTube with the selected video.

• It waits a bit to ensure the YouTube app has loaded - this part is admittedly guesswork, as there's no API feedback about app state.

• Finally, it navigates the YouTube guest mode menu and starts playback.

Here’s the function that brings it all together:

def configure_tv_play_video(): 
    if not tv_online():
        turn_on_tv()

        # Wait for the TV to appear on the network, try up to 10 times. 
        for _ in range(10):
            if tv_online():
                break
            else:
                time.sleep(3)

    # Establish Connection With LG WebOS TV
    client = connect_to_tv()

    # Set volume to something low 
    media = MediaControl(client)
    media.set_volume(8)

    # System control allows notifications to be sent to TV
    system = SystemControl(client)
    system.notify("Connected to the TV. Youtube will launch shortly.")  

    # Get a youtube video to watch 
    video = get_youtube_video()

    # Launch YouTube on the TV with the selected video. 
    app = ApplicationControl(client)
    apps = app.list_apps()
    yt = [x for x in apps if "youtube" in x["title"].lower()][0]
    app.launch(yt, content_id=video)

    # Wait for the app to launch
    system.notify("Waiting 15 seconds for Youtube to launch...")  
    time.sleep(15)

    # Select YouTube guest mode 
    inp = InputControl(client)
    inp.connect_input()
    inp.down()
    time.sleep(1)
    inp.ok()
    inp.disconnect_input()

    system.notify("Video should now be playing. Enjoy!")

Getting an Unwatched YouTube Video

Generic YouTube Channel Class

To retrieve a new video from the user’s favourite YouTube channel, I implemented a class that uses the yt_dlp library. This lets me scrape the necessary video metadata directly from a given YouTube channel.

The class, YoutubeChannel, is initialised with the name of the channel. When the scrape_video_urls method is called, it fetches all video entries from the channel's /videos page. For each entry, I check whether the video meets a defined acceptance criteria using _accept_video. By default the function always returns True, but I expect to overwrite this later when implementing a subclass for a specific channel.

Each video ID is added to a dictionary where the key is the ID and the value is a boolean indicating whether the video has been "watched" or not (we start by assuming all videos are unwatched). When a video is selected via get_unwatched_video_url, it randomly picks an unwatched one from the list, marks it as watched, and returns the ID so it can be launched in YouTube.

Here’s the class implementation:

import yt_dlp
import random
from urllib.parse import urlparse, parse_qs

class YoutubeChannel: 
    def __init__(self, channel_name):
        self.channel_name = channel_name
        self.channel_url = f'https://www.youtube.com/@{channel_name}/videos'
        self.video_urls = dict()

    def scrape_video_urls(self):
        ydl_opts = {
            'quiet': True,
            'extract_flat': True,  # Do not download, just get metadata
            'skip_download': True,
        }

        with yt_dlp.YoutubeDL(ydl_opts) as ydl:
            info = ydl.extract_info(self.channel_url, download=False)
            entries = info.get('entries', [])

            for entry in entries:
                if self._accept_video(entry) and entry['url'] not in self.video_urls:
                    parsed_url = urlparse(entry['url'])
                    query_params = parse_qs(parsed_url.query)
                    video_id = query_params.get('v', [None])[0]
                    self.video_urls[video_id] = False # False means not watched yet

    def get_video_urls(self):
        return list(self.video_urls.keys())

    def get_unwatched_video_urls(self):
        unwatched = [url for url, watched in self.video_urls.items() if not watched]

        # If there are no unwatched videos, return all video URLs
        if len(unwatched) == 0:
            return self.get_video_urls()

        return unwatched

    def get_unwatched_video_url(self):
        unwatched = self.get_unwatched_video_urls()
        chosen_video = random.choice(unwatched)
        self.video_urls[chosen_video] = True
        return chosen_video

    def _accept_video(self, entry):
         return True

Dedicated YouTube Channel Subclass

My family members favourite channel is 48 Hours, so a dedicated subclass is implemented. I overwrite the _accept_video function so that only full episodes and no parts are loaded. As the videos are chosen at random, landing on Part 1 of a multi-part story, without Part 2 automatically following, and no easy way to find it manually, would be maddening. So, I’ve removed that possibility with this filter.

class YoutubeChannel48Hours(YoutubeChannel):
    def __init__(self):
        super().__init__('48hours')

    def _accept_video(self, entry):
        return "Full Episode" in entry['title'] and "Part" not in entry['title']

Persistent Storage and Getting Videos

Keen readers may have noticed that, up to this point, there hasn’t been any persistent storage of the YouTube videos we've scraped or whether they’ve been watched. I handle this in the get_youtube_video function using Python’s built-in pickle library. This allows the entire object, complete with its internal state and watched/unwatched video list, to be serialised to disk and reloaded on future runs.

When the function is called, it first checks for the presence of a previously saved file. If the file exists, it loads the YoutubeChannel48Hours instance from disk, giving us instant access to the existing video list and watched statuses. If not, it creates a fresh instance of the class and populates it by scraping the channel again.

In either case, it then selects a random unwatched video. After that, the updated object, now with one more video marked as watched, is serialised and saved for next time. I thought this was a neat solution: no need to manually parse or manage separate data files, and it all just works with a couple of lines of code.

import pickle

def get_youtube_video():
    # Get a youtube video to watch 
    if os.path.exists('ytc_48_hours.pickle'):
        with open('ytc_48_hours.pickle', 'rb') as f:
            ytc_48_hours = pickle.load(f)
    else:
        ytc_48_hours = YoutubeChannel48Hours()
        ytc_48_hours.scrape_video_urls()

    video = ytc_48_hours.get_unwatched_video_url()

    # Pickle youtube channel object(s) to file
    with open('ytc_48_hours.pickle', 'wb') as f:
        pickle.dump(ytc_48_hours, f)

    return video

Controlling from a Button

The final step was making everything work with a single button press. I wanted this to be as simple as possible for my family member—no apps, no remotes, just a physical button.

From previous experiments, I had a 433 MHz wireless button and a compatible receiver lying around. Since everything else was already running on a Raspberry Pi, it made perfect sense to hook the receiver up to it and listen for the specific signal sent by this button. The Raspberry Pi has General-Purpose Input/Output (GPIO) pins that make this possible.

I wrote a small script using the rpi-rf library to monitor incoming RF codes. When it detects the code sent by this particular button, it calls the configure_tv_play_video() function and kicks off the whole process.

import time

from rpi_rf import RFDevice
from webos import configure_tv_play_video

RX_PIN = 27  
rfdevice = RFDevice(RX_PIN)
rfdevice.enable_rx()
timestamp = None

print("Listening for RF signals...")

try:
    while True:
        if rfdevice.rx_code_timestamp != timestamp:
            timestamp = rfdevice.rx_code_timestamp
            received_code = rfdevice.rx_code
            print(f"Received code: {received_code}")

            if received_code == 13739617: # the code sent from my RF button. 
                try:
                    configure_tv_play_video()
                except Exception as e:
                    print(f"Couldn't configure tv and play video: {e}")

        time.sleep(0.01)

except KeyboardInterrupt:
    rfdevice.cleanup()

Complete Code

The complete code for my project can be found in this GitHub repository:

Leave to Remain

With this setup, my elderly family member, my digital immigrant, has finally crossed into the land of modern technology. With a single button press, the TV powers on (if it isn’t already), launches YouTube, and starts playing a new video from their favourite channel. If the TV is already on, it simply loads another video.

The one-button interface transforms what was once an intimidating, multi-step process into something effortless and accessible. It’s been amazing to see them confidently engage with a platform that used to feel completely out of reach. In a small but meaningful way, they now feel more connected to the digital world—and that, to me, makes the whole project worth it.

Here is an example of a traditionally implemented Person class, and its equivalent data class implementation below. You can see that the data class implementation is much more concise.

class Person():
    def __init__(self, forename, surname, age, sex):
        self.forename = forename
        self.surname = surname
        self.age = age
        self.sex = sex

@dataclass
class Person:
    forename: str
    surname: str
    age: int
    sex: str

Learning About Python Data Classes

I looked at two articles, and the official documentation:

• Python Documentation

• How to Use Python Data Classes in 2023 (A Beginner’s Guide)

• Python Data Classes: A Comprehensive Tutorial

I won't go into reproducing the examples here, but I recommend checking out these articles for some nice demonstrations. They provide valuable insights and practical examples that will give you a deeper understanding of how to use Python data classes effectively.

Thoughts on Python Data Classes

After learning a bit about this Python feature, I had a think about some pros and cons.

Pro: Reduced Boilerplate

One of the main advantages of using data classes is the reduction of boilerplate code. Without data classes, you would need to manually write methods like:

• __init__() that is used initialise attributes;

• __repr__() that is used to define the string representation for a class;

• __eq__() that is used for comparison.

With a Python data class, these methods are automatically generated, giving developers a quick and robust set of core of features for the class. Additional features may be enabled, and any may be overwritten in the traditional way - by overriding the method implementation.

Pro: Improved Readability

I think that data classes enhance code readability by making the structure and purpose of the classes clearer. The class definition itself is compact, and the use of the @dataclass decorator signals that the class is meant for storing data, making its purpose clear. Furthermore, the generated __repr__() method provides a readable string representation of the object, which is immediately useful for debugging.

Con: Limited Utility

While the automatic features of data classes, such as reducing boilerplate code and enhancing readability, are appealing, they do have limitations. Data classes are primarily designed to represent simple data structures. If you try to extend their use to more complex functionality or logic, they can quickly become cumbersome. Initially, a data class might seem like the perfect fit, but as the class evolves and requires more complex behaviour or internal state management, you may find yourself fighting against the design of the data class. In such cases, it could become more practical to refactor the class into a traditional class implementation, which may involve more effort but offers greater flexibility. While it’s tempting to use data classes for their convenience and automation, it's important to ensure that your use case aligns with their intended purpose. If the class is genuinely meant for simple data storage, data classes are a great choice, but if you're venturing into more complex logic, you might eventually find them a poor fit.

Unlike humans, who continuously seek, absorb, and apply new information, Large Language Models (LLMs) in their basic form are frozen in time. Their knowledge is limited to what they learned during their last pre-training. They can’t naturally update themselves with new events or discoveries.

Pre-training is an expensive, resource-heavy process. It involves collecting vast amounts of text data and refining it into a statistical model that recognises patterns. Rather than storing knowledge like a database, the model learns relationships between words, allowing it to predict the next token in a sequence. While this creates the illusion of understanding, LLMs don’t actually "reason" or "remember" as humans do - they recognise patterns based on probability.

Since pre-training happens infrequently, LLMs have a knowledge cutoff. For example, if a model was last trained in November 2024, it won’t “know” anything beyond that date. So, if an LLM relies only on pre-training, it won’t be able to answer questions about recent news, ongoing events, or the latest scientific discoveries.

LLMs are strongest when answering questions about frequently discussed topics. If something is widely covered on the internet (like general science concepts, historical events, or common knowledge) it’s likely the model will provide accurate answers. But for rarer, less-documented information, its knowledge is hazy. For example, an LLM will confidently tell you the capital of Scotland or the laws of physics. But if you ask about a little-known research paper, a local news story, or an obscure hobby, it may struggle. It may even make something up.

A Familiar Fix

As humans, when we encounter something we don’t know, we turn to external resources like Google, instruction manuals, or documentation. For complex tasks like advanced math, we rely on specialised tools such as calculators. We recognise our own limitations and use these tools to help us reach the right answer.

LLMs can follow the same approach. By integrating real-time search capabilities, accessing external databases, or working alongside other specialised tools, they can overcome the gaps in their knowledge. Instead of relying solely on what they already know, they can be designed to seek out the right information, just like we do. One way of doing this is called Retrieval-Augmented Generation (RAG), which I have explored for the first time.

Retrieval-Augmented Generation (RAG)

In the simplest interactions with an LLM, you provide text as input, which is then tokenised and placed into the model's context window; the LLMs working memory. The LLM processes this input using its fixed model and generates a response solely based on learned patterns and probabilities. With RAG, your input (or prompt) is first processed to identify relevant external content from sources like databases, documents, or the internet. This retrieved information is then combined with your prompt and placed into the model's working memory. By incorporating this additional context, the LLM generates a more informed and accurate response, grounded in the relevant data.

I decided to explore the use-case where a set of documents serve as a knowledge base for a simple AI agent capable of answering questions about the documentation. My goal was to see if I could host an LLM locally on my laptop and have the agent function entirely offline, without relying on any paid models.

Creating the RAG Tool with LangChain

I did a bit of research, and watched a couple of YouTube tutorials around the topic.

The first video provided a great overview of how to use your own documents effectively with an LLM. While a single document or a small set may sometimes be sufficient, handling larger information sets requires a more advanced approach like RAG.

The next video introduced Ollama, a tool that allows you to run and manage large language models (LLMs) locally on your device, making it easier to experiment without relying on cloud services. It also covered LangChain and walked through a tutorial on building a simple RAG-based LLM agent using PDFs as a knowledge base. The process seemed straightforward and showed how easy it would be to host my own LLM instance on my laptop.

With these videos, and following the first part of a LangChain tutorial, I was able to implement a simple local agent that is able to answer questions on my own documents.

Choosing a Model

The first step in the build was to select a LLM and embeddings model. The LLM is the part that will take our query, context and produce a human-like response. The embedding model will be responsible for taking chunks of our documentation and producing embeddings.

Embeddings are a powerful way to represent words as numerical vectors in a high-dimensional space, capturing their semantic relationships. As a simple example, consider the words "king", "queen", "man", and "woman". An embedding model might assign them vectors:

$$\text{king} \rightarrow \begin{bmatrix} 0.8 \\ 0.6 \\ 0.7 \end{bmatrix}, \quad \text{queen} \rightarrow \begin{bmatrix} 0.8 \\ 0.7 \\ 0.6 \end{bmatrix}, \quad \text{man} \rightarrow \begin{bmatrix} 0.9 \\ 0.5 \\ 0.8 \end{bmatrix}, \quad \text{woman} \rightarrow \begin{bmatrix} 0.9 \\ 0.6 \\ 0.7 \end{bmatrix}$$

These numerical representations allow the model to capture meaningful relationships. Vector arithmetic can then be used to work with our text. For example, by performing the operation,

$$\text{king} - \text{man} + \text{woman}$$

$$\begin{bmatrix} 0.8 \\ 0.6 \\ 0.7 \end{bmatrix} - \begin{bmatrix} 0.9 \\ 0.5 \\ 0.8 \end{bmatrix} + \begin{bmatrix} 0.9 \\ 0.6 \\ 0.7 \end{bmatrix} = \begin{bmatrix} 0.8 \\ 0.7 \\ 0.6 \end{bmatrix}$$

the result is our vector, our embedding, of the word queen.

For this simple local agent I chose llama3.2 as my LLM and nomic-embed-text as my embeddings model. Both were available through Ollama.

# Select chat model
llm = OllamaLLM(model="llama3.2")

# Select embeddings model
embeddings = OllamaEmbeddings(model="nomic-embed-text")

Selecting a Vector Store

I then chose Chroma as the vector store - a database that stores the embeddings. This is what was used in the second video. I didn’t like the idea of storing the embeddings in memory (suggested in the LangChain tutorial) where the embeddings would have to be re-created each time the program was run. The database provides persistent storage of the embeddings. Other options were available.

# Select vector store where embeddings are stored
vector_store_dir = "chroma_db"
vector_store = Chroma(embedding_function=embeddings, persist_directory=vector_store_dir)

Loading Documents

Information may be loaded from many sources (websites, Word documents, PDFs) using the corresponding LangChain document loaders. I was interested in loading all of the PDFs in a directory, so I use the PyPDFDirectoryLoader.

Loaders typically have .load() and .lazy_load() functions. The lazy load function returns a generator which I think can be helpful in applications that require memory and performance optimisation.

By default, the PDF document loader will not parse any images. I looked into enabling this feature with Tesseract as I liked the idea of the content of diagrams in my documentation being utilised. However, the specific PDFs I was interested in using had images that were unsupported by this optical character recognition (OCR) library. I proceeded without this potentially useful feature.

Once loaded, we get a list of documents. Each element in this list is a page from our PDFs. For example, if the folder contained a PDF with 2 pages, and another with 6, we would get a python list with 8 elements; our 8 pages.

# Meaningful name to describe the data being used in the vector store
collection_name = "pdf_docs"

# Loader for all of the documents in the pdf_data directory
pdf_loader = PyPDFDirectoryLoader(collection_name)
docs = pdf_loader.load()

Splitting Documents

Inserting entire pages of documents into a LLMs context window would be inefficient. It would be unlikely that an entire page would be relevant to your prompt, and models may struggle to find relevant content. As a next step we split our documents (our pages) into smaller chunks.

In this case the RecursiveCharacterSplitter is used to split pages into 1000 character chunks, with an overlap of 200 characters. These values wont be arbitrary, but I have not experimented with larger or smaller chunks sizes, and simply took the values from the LangChain tutorial. It would be interesting to see the effect that different chunk sizes and overlap had.

As a concrete example, I gave my agent two PDFs totalling 231 pages, so 231 documents. This was split into 681 smaller chunks. Note that the chunks have useful meta data like the name of the original document they came from.

# Split the documents
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,       # chunk size (characters)
    chunk_overlap=200,     # chunk overlap (characters)
    add_start_index=True,  # track index in original document
)
all_splits = text_splitter.split_documents(docs)

Adding New Splits to the Database

I was keen for the embeddings not to be re-generated with each run of my agent. If I was using a paid-for model to generate embeddings, I would be concerned with the needless expense of doing this repeatedly. For free models, I am more concerned with the time it takes to generate the embeddings constantly, and the time this takes.

I was able to follow documentation on LangChain Indexing that outlined how to efficiently manage embeddings. I provide the indexing function with document chunks, a record_manager that allows inspection of the database contents, and a Chroma vector_store configured with our chosen embedding model. I opt for the 'full' cleanup mode, which ensures efficient management by:

• De-duplicating content

• Removing chunks whose source PDFs no longer exist

• Updating chunks when their original PDFs have changed

• Retaining chunks that remain unchanged in both the database and the PDF folder

The source_id_key is set to 'source', ensuring that each chunk is associated with its original PDF filename for tracking and management.

# Used to manage the contents of the Chroma vector store. 
record_manager = SQLRecordManager(namespace, db_url=f"sqlite:///{vector_store_dir}/chroma.db")
record_manager.create_schema()

# Add new or updated chunk embeddings to the database 
index_result = index(all_splits, record_manager, vector_store, cleanup='full', source_id_key='source')

Creating the Prompt

I created an prompt that will be used whenever the agent is asked a question. Here we ‘configure’ the LLM to be an agent that answers a question based on the content. The agent should answer honestly if the answer cannot be derived from the content. The user’s question, and the retrieved context, are substituted at runtime into the {question} and {context} parts of the prompt.

prompt = PromptTemplate.from_template("You are an assistant for question-answering tasks. \
                                       Use the following pieces of retrieved context to answer the question. \
                                       If you don't know the answer, say that you don't know.\
                                       \nQuestion: {question} \
                                       \nContext: {context} \nAnswer:")

Managing State

From what I understand, LangGraph applications manage state through a typed dictionary. The intention is for the input, intermediate and output data to be kept track of with this. For our application, the question (input data), context (intermediate data transferred between steps), and answer (output data) are managed with this structure.

# Keep track of the state of the input question, retireved context and genrated answer 
class State(TypedDict):
    question: str
    context: List[Document]
    answer: str

Application Steps

Next we define the steps to be taken, LangChain refers to these as nodes. In our case we have a retrieval and generate node. Both steps are provided with the state of our application, and so have access to the question, context and answer.

The retrieve step takes the users question and performs a similarity search of our vector store. The result of the search is the four most relevant chunks of our documents. Four just happens to be the default value for the number of chunks to return and can be increased at will. I have not experimented with this.

This output of this node forms the context for our user’s question.

def retrieve(state: State):
    # Search the vector store for the most relevant documents to the input question
    retrieved_docs = vector_store.similarity_search(state["question"])
    # This forms the context for our search
    return {"context": retrieved_docs}

The generate step fills our prompt outline with the users question and the context that was retrieved. This is then passed as the prompt to the LLM which provides a response to the user’s question.

def generate(state: State):
    # Join all the documents in the context into a single string
    docs_content = "\n\n".join(doc.page_content for doc in state["context"])
    # Generate the answer to the input question
    messages = prompt.invoke({"question": state["question"], "context": docs_content})
    # Ask LLM to answer the question
    response = llm.invoke(messages)
    # LLM response forms our answer
    return {"answer": response}

Connecting the Steps

These individual steps (or nodes) are then compiled into a graph forming a sequence of steps. Here a StateGraph, “a graph whose nodes communicate by reading and writing to a shared state,” is used. Our application starts with the retrieval step and then performs the generation step.

# Compile application into a graph object that connects the retrieval and generation steps into single sequence. 
graph_builder = StateGraph(State).add_sequence([retrieve, generate])
graph_builder.add_edge(START, "retrieve")
graph = graph_builder.compile()

Asking the Question

We can then use our graph to form a very basic application that repeatedly asks for the user to ask a question relating to the documents provided. LangGraphs support multiple invocation methods, in this case I have chosen to use the ‘stream’ method that lets me get the output of each step as the graph is executed.

# Ask user to enter a question
question = input("Please enter a question relating to the documents you have provided:")

while question:
    # Crudely print the context and the answer to the question.
    for step in graph.stream({"question": question}, stream_mode="updates"):
        print(f"{step}\n\n----------------\n")

    # Ask user to enter another question
    question = input("Please enter a question relating to the documents you have provided:")

Example Output

As a test I created a knowledge-base of two PDF’s containing the instructions to two fictional board games; generated by ChatGPT. You can see, from the output below, the question that I asked, the 4 most relevant chunks of context that were retrieved (alongside their metadata), and the useful (correct) answer that the application was able to provide based on the context.

{
  "question": "In realm of the arcane lords, what determines the order of play?",
  "context": [
    {
      "content": "Realm of the Arcane Lords: A Game of Magic, Strategy, and Diplomacy Objective of the Game: The ultimate goal of \"Realm of the Arcane Lords\" is to ascend to the title of Supreme Arcane Lord by accumulating Arcane Power, forming strategic alliances, and tactically outmaneuvering your opponents across the mystical lands of Eldorin. A player wins by either amassing 100 Arcane Points (AP) or by securing dominance over three of the four Ancient Elemental Obelisks scattered across the game board. Components: \u2022 1 Large Game Board depicting the world of Eldorin, divided into 48 Territories \u2022 5 Sets of 15 Territory Control Markers per player \u2022 200 Arcane Power Tokens (AP) \u2022 4 Elemental Obelisk Tokens (Earth, Fire, Water, Air) \u2022 120 Spell Cards (divided into Attack, Defense, Support, and Forbidden Magic categories) \u2022 1 20-sided die (D20) and 3 6-sided dice (D6) \u2022 100 Resource Cards (Gold, Crystals, Herbs, and Relics) \u2022 6 Player Faction Boards representing different mystical orders \u2022 50 Miniature",
      "metadata": {
        "creationdate": "D:20250322185747Z00'00'",
        "creator": "PyPDF",
        "moddate": "D:20250322185747Z00'00'",
        "page": 0,
        "page_label": "1",
        "producer": "macOS Version 14.6.1 (Build 23G93) Quartz PDFContext",
        "source": "pdf_docs/Realm of the Arcane Lords.pdf",
        "start_index": 0,
        "total_pages": 2
      }
    },
    {
      "content": "territories. \u2022 Forbidden Magic Spells can only be used once per game and carry significant risk (e.g., backfiring catastrophically). \u2022 Players may attempt to Bargain with the Elders\u2014a mysterious game mechanic where a player rolls three D6 dice and consults the Elder\u2019s Fate Table to receive a boon or curse. Final Thoughts: \"Realm of the Arcane Lords\" is a game that rewards strategy, diplomacy, and careful planning. Players must weigh the risks of confrontation against the benefits of alliance and resource gathering. Every choice carries weight, and only the most cunning and powerful will rise to claim the title of Supreme Arcane Lord!",
      "metadata": {
        "creationdate": "D:20250322185747Z00'00'",
        "creator": "PyPDF",
        "moddate": "D:20250322185747Z00'00'",
        "page": 1,
        "page_label": "2",
        "producer": "macOS Version 14.6.1 (Build 23G93) Quartz PDFContext",
        "source": "pdf_docs/Realm of the Arcane Lords.pdf",
        "start_index": 794,
        "total_pages": 2
      }
    },
    {
      "content": "8. Players take turns placing one Miniature Wizard onto an unoccupied territory until all starting positions are chosen. Gameplay Mechanics: Each turn consists of four phases: 1. The Arcane Planning Phase: o Players may exchange resources, form temporary alliances, or trade Spell Cards. o Players may activate passive abilities from their faction board. o Players may draft one new Spell Card from the deck. 2. The Tactical Movement Phase: o Players may move their wizards across the board, with a maximum movement of two spaces per turn. o Entering an opponent-controlled territory initiates a Duel of Arcane Might (combat sequence). 3. The Duel of Arcane Might: o The attacker rolls a D20 and adds any applicable bonuses from Spell Cards, resources, or faction abilities.",
      "metadata": {
        "creationdate": "D:20250322185747Z00'00'",
        "creator": "PyPDF",
        "moddate": "D:20250322185747Z00'00'",
        "page": 0,
        "page_label": "1",
        "producer": "macOS Version 14.6.1 (Build 23G93) Quartz PDFContext",
        "source": "pdf_docs/Realm of the Arcane Lords.pdf",
        "start_index": 1602,
        "total_pages": 2
      }
    },
    {
      "content": "categories) \u2022 1 20-sided die (D20) and 3 6-sided dice (D6) \u2022 100 Resource Cards (Gold, Crystals, Herbs, and Relics) \u2022 6 Player Faction Boards representing different mystical orders \u2022 50 Miniature Wizards representing each player\u2019s controlled sorcerers \u2022 1 Rulebook with extended lore Setup: 1. Each player selects a mystical order and receives the corresponding Faction Board. 2. Players each take 15 Territory Control Markers in their faction\u2019s color. 3. Shuffle and place the Spell Cards in their respective piles. 4. Shuffle the Resource Deck and place it near the board. 5. Distribute 10 Arcane Power Tokens to each player. 6. Randomly place the four Elemental Obelisk Tokens in different quadrants of the board. 7. Players roll the D20 to determine the order of play; highest roll goes first. 8. Players take turns placing one Miniature Wizard onto an unoccupied territory until all starting positions are chosen. Gameplay Mechanics: Each turn consists of four phases: 1. The Arcane Planning",
      "metadata": {
        "creationdate": "D:20250322185747Z00'00'",
        "creator": "PyPDF",
        "moddate": "D:20250322185747Z00'00'",
        "page": 0,
        "page_label": "1",
        "producer": "macOS Version 14.6.1 (Build 23G93) Quartz PDFContext",
        "source": "pdf_docs/Realm of the Arcane Lords.pdf",
        "start_index": 804,
        "total_pages": 2
      }
    }
  ],
  "answer": "According to the context, players roll a D20 to determine the order of play; the highest roll goes first."
}

Asking the LLM the same question without any context of our made up game, of course, results in a made-up answer.

“In realm of the arcane lords, what determines the order of play? In the board game "Realm of the Arcane Lords", the order of play is determined by a unique mechanism called the "Arcane Order" track. Each player has three Arcane Points (AP) that they can use to determine their turn. The AP are distributed at the beginning of the game, and players take turns playing cards from their deck in a specific sequence based on the number of AP they have available.“

RAG Recap

I've really enjoyed learning about RAG, and I see it as a powerful tool that complements. For me, its biggest potential lies in navigating complex proprietary software documentation, things like company policies, dense technical manuals, and inaccessible stacks of information. With RAG, all of this could become easily searchable through a simple Q&A session in a more mature application.