Last week our staff engineer Will Hakim wrote about his learnings from building with artificial intelligence and large language models.  One of his most important observations is that AI applications thrive not just based on technology, but from (in his words) understanding a customer’s workflow and fitting AI into it.

We can also add a higher level of abstraction (or, two levels of abstraction) to his observation.  Building with artificial intelligence in order to enable better business and strategy decision-making in energy transition requires embodying two other attributes as well.  The first is accuracy; the second is customization. 

Accuracy is table stakes in providing information as a service.  As our advisor Sharon Reishus reminds us, pulling complex, poorly-indexed information from disparate sources into a coherent accessible framework is great, but if it isn’t as accurate as human researchers, it’s not worth much.  Her estimate is that human researchers are 99% accurate when extracting structured information from unstructured context, which is a reasonable aim for a technology platform at first.

Many of you reading this will know that large language models are not always accurate on every matter, and that their differential accuracy may not always seem entirely sensible.  As an example, I use ChatGPT 4o frequently to create CSV files of data from messy text sources, and it does so quickly and accurately.  When I ask it to do the same by plotting information that has already been visualized into a clearly legible chart, such two-axis line or column chart, it almost always fails. It creates a convincing-looking data set which, if I did not observe closely, might lull me into believing it was correct. 

What Halcyon continues to learn from working with large language models, however, is promising.  It is not so much that LLMs are inherently 100% accurate at any moment in time, but rather that their rate of improvement shows us a path to enabling 99% accuracy in the future.  Delivering consistent accuracy over time requires not just inherent improvements to foundation models, but also our own technology build-out as well, to provide quality assurance and quality control on the increasing volume of information that models provide to us. 

The second attribute to embody is customization.  Accurate humans at research and consulting firms are capable of extracting, harmonizing, packaging, and delivering key data from messy complex contexts. They provide significant value to customers in doing so.  However (and speaking from experience) even very accurate humans run into the limits of time and space in their work.  That is why analysis is often packaged in a static fashion, and released on a set cadence: it is the only way to manage what could be an infinitely multi-variate combination of information sources and choices. 

Customization - or to put it more plainly, choice - is another area that we believe will move at technology speed thanks to large language models. Rapidly-improving LLMs should enable both greater customization of requested information from disparate sources, and greater velocity in refreshing and harmonizing it as well.  Combining breadth and scope with a continually growing programmatic understanding of how information sources relate to each other will ideally make customer choice inherent in what we build.  Choice and customization should not be features; they should be expectations. 

Soon, Halcyon will have products available to test widely.  If you’re not already signed up for our product waitlist, you can do so here and be the first to hear more.

Comments or questions? We’d love to hear from you - sayhi@halcyon.eco, or find us on LinkedIn and Twitter