The researchers increased success rates by as much as 27% with simple prompt engineering.
Getting ChatGPT to operate autonomously within the confines of an operating system has proven a difficult task for numerous reasons, but a team composed of scientists from Microsoft Research and Peking University may have figured out the secret sauce.

The team conducted a study to determine why artificial intelligence (AI) large language models (LLMs) such as GPT-4 fail at tasks requiring the manipulation of an operating system.

State of the art systems such as ChatGPT running on GPT-4 set the benchmark for generative tasks such as drafting an email or writing a poem. But getting them to act as agents within a general environment poses a significant challenge.

Traditionally, AI models are trained to explore through reinforcement learning in a virtual environment. AI developers have used modified versions of popular video games such as Super Mario Bros. and Minecraft to “teach” models concepts such as self-guided exploration and goal seeking.

But operations systems are an all-together different playground for AI models. As agents, performing functions within an OS often presents as a multimodal challenge requiring the exchange of information between different components, programs, and applications.

Generally speaking, the reinforcement training approach requires trial and error. However, as anyone who has entered their password incorrectly too many times, or forgotten which shortcuts work in which apps knows, data can easily be lost when using such an approach in an operating system environment.

Source - Coin Telegraph