How does it actually save me time?

IThe problem is that LLMs rely on having extremely large volumes of source material in order to generate meaningful text. For many use cases the model training data is sufficient and represented by default in the model weights used for inference.

For many more use cases specific context to the task at hand is not present in the training data and it therefore can't be used. For example if one was using an LLM to give feedback on a book they were writing the LLM has no context for what that person has written.

There's a subset of problems where there's an easy solution involving just providing the context of what you want from the LLM - easy example would be re-wording an email, it's trivial to just provide the existing email to the LLM. However there is another subset of problems with two potentially overlapping issues, and this is ultimately the source of the problems software engineers face on daily basis.

Those issues are either 1) the source/context an LLM needs is buried in some kind of data store eg perhaps 100 different files on my computer and/or 2) the source is too large for an LLM to handle and it needs to be pruned/curated prior to providing it to the LLM.

What does this mean? Software Engineers can't utilise the, frankly absurd, productivity enhancing features of an LLM for tasks  with complicated and/or lengthy context without significant effort into finding and selecting the correct subset of context to provide the LLM for a specific task.

The effort, cognitive load, and time taken of doing that increases exponentially with the size of the task. At some point it becomes such a burden than reverting to not using an LLM for assistance becomes the best option because it's simply too hard to curate the context required to get results that will actually be helpful.

This is extremely frustrating and disappointing because when a task's scale allows for easy context management for the LLM, it's easy to see 10x boosts in productivity for some things. Then for very complex tasks using an LLM ends up actually reducing productivity because it's just too hard to curate the context.

There is a focus above on writing/research-like tasks, mostly because it's easy to picture those in examples. This problem applies equally to more technical automation workflows that involve LLMs. In those cases where an LLM is used a decision making component providing the correct context is crucial and in an automated environment generally becomes less reliable as the number of options increases.

What tasks would I use it for?

• Build a Knowledge Graph or Semantic model of your private documents

• Simple research and text preparation tasks

• Or more technical tasks,

• Deploy as an agent

• A piece in a architecture pipeline

• Decision making component

• LLM super prompter - context enhancer

How can I make curating the context easier?

The preparation of files for an LLM becomes a lengthy, time consuming task.

When you use Leonata curating the context is easier in order to increase the scope in which the productivity enhancement of LLM assistance is attainable; and another option is to make the automated selection of context more reliable for LLM enhanced AI decision making.

What really is a Large Language model?

Large Language Models can also be known as translation tools, they essentially pattern match by….

They are adept at telling you what you want to hear but fall down at making decisions or using critical thinking.

They are best suited to tasks that require generating content for non-critical, no important tasks where the reward is high and the risk is low.