AI Research

At its core, an automation system is a programming environment. As much as we aim to abstract away low-level details, but those details often creep back, through control constructs, types, or the quirks of web APIs. How can we let users access the full expressiveness of a programming system, without being programmers themselves?

An old story tells of a technician who was hired to fix a machine. They came, turned a single screw, and then billed a large sum. The large sum wasn’t for the work itself, but for knowing which screw to turn. Indeed, human work, and office knowledge work specifically, is often about the person’s own knowledge and decision making, which is part of the workflow and can be expressed in simple rules, API calls, or clicks. If someone wishes to automate their workflow, they need to be able to describe that knowledge and decision process to the machine. Hence, how can users encode those steps of the workflow in a way that can be automated?

With the advent of large language models, it is now possible to quickly build high quality semantic parsers even for complex tasks. Yet, accuracy is not perfect, and natural language is inherently ambiguous.

How can users be confident that their prompt was understood correctly and that the automation will implement their desired goal, down to their intended details?

Beyond accuracy, the main limitation of using natural language as input modality is that it requires the user to formulate a clear goal in writing. Meanwhile, many users are out there performing tasks by hand day in and day out, without even realizing that those tasks can be automated.

‍Can we automatically identify what parts of the user’s workflow can be automated, and then suggest the right automation to do so?

There are many thousands of services out there that users would like to interface with. Many of those services have public APIs, but those APIs are messy, bespoke, inconsistent. At the same time, the code to access these services is in large part boilerplace, which LLMs are very good at.

Therefore, can we use LLMs to automatically generate the code to access an API from Bardeen?

Many automation tools have 5,000+ services, and Alexa 200,000+ skills, the web has billions of websites that have no APIs, and users want to make use of those websites in their workflows.

‍How can we enable users to interact with all those websites in their automations?