That artificial intelligence (AI) promises to leave a profound mark on companies is difficult to doubt today, even despite the youth of tools such as Stable Diffusion, Midjourney V5 or GPT-4. It is not a matter of hunches or suspicions. Its impact is already being felt in certain companies and trades and there are even studies, such as the one published in March by researchers from OpenResearch, the University of Pennsylvania and Open AI, father of ChatGPT and DALL-AE, which acknowledge that their models can affect to thousands of jobs.
The big unknown is… What scenarios open up in the interaction between workers and AI? Can it become an ally, make us more efficient? at MIT they have already studied the matter. And his conclusions are revealing.
What did your study consist of? Basically, in checking how ChatGPT affects our productivity as workers. At least that of a sample group that is dedicated to preparing commercial documents. To do this, Shakked Noy and Whitney Zhang of MIT brought together 444 professionals with experience in different tasks, such as salespeople, editors, analysts and technicians in the Human Resources area. Then they asked each of them to write two texts in their respective fields, such as reports, analysis plans or press releases, tasks similar to those they carry out for their company on a day-to-day basis.
The first document was written routinely, with their usual resources. Things started to get interesting with the second: half were allowed to use ChatGPT and the remaining 222 returned to work in a “normal” way, with their traditional resources and without any assistance from artificial intelligence. With all the material ready, the documents were evaluated by three professionals who assigned them a grade between 1 and 7. When examining them, of course, they did not know which ones had been made with AI.
And what did they check? That the employees who had had the help of ChatGPT stood out in two points: speed and quality. When looking at the first documents, all produced without AI support, the results of the two test groups were similar, but things changed when 222 of the workers were able to get their hands on the engine of Open AI. In that case there were differences.
Employees who had handled AI handed in their jobs in 17 minutes, well below the 27 of the colleagues who did not have the help of algorithms. If this result is transferred to a whole working day, it means that a professional assisted by artificial intelligence would be able to produce a total of 28.3 documents compared to 17.7 for another deprived of that help. The difference in terms of productivity is overwhelming: 59%.
But… And the quality of the work? In that they also excelled. More speed sometimes leads to a decrease in quality, but this premise was not met in the test of the MIT. When it was their turn to evaluate the documents, works of unknown origin, the examiners assigned an average of 4.5 to those that had been carried out with the help of ChatGPT compared to 3.8 received by those carried out without AI. If we take into account that the scale went from 1 to 7, the difference is noticeable.
The data is relevant for another reason: most of the employees who used ChatGPT had not handled it before, so they lacked experience. To be more precise, only 30% of the participants had used it before. The nuance is quite important because those who handled ChatGPT did not have a prior learning curve, usually a fundamental requirement to take full advantage of a new tool.
And what is the explanation? As details Nielsen Norman Group, using the tool reduces skill imbalances. Those who benefited the most in the final score were the workers who had obtained a lower grade in the first phase of the test and used the AI in the second. Another of the keys is that the management of ChatGPT changed the way in which employees invested their time in carrying out tasks.
Without the engine they dedicated 25% to the brainstorming, 50% to the writing of the draft and the remaining 25% to its edition to achieve the final result. When AI entered the equation, the amount of time spent creating drafts more than halved and the amount of time spent on “polishing” doubled. What does that mean? A shorter total time and more attention in the final phase, focused precisely on perfecting the result. The investment of minutes in the brainstorming it was also lower, although the cut falls within the margin of error contemplated in the study.
Are the results relevant? yes, although as recognized in Nielsen Norman GroupThey should be handled with some caution. The reason: the limitations of the study, focused on a very specific professional profile and on a task that is also quite specific. Writing each document without the help of AI ultimately required less than half an hour from the employees. The conclusions of the report are significant, but they leave doubts drooping, such as the impact of AI on other professionals, different professional categories and more extensive tasks.
Cover image: Emiliano Vittoriosi (Unsplash)
In Xataka: AI has already generated the new fashionable job (with a big salary) in the technology industry: “prompt engineer”
