Hybrid working works: huge study reveals no drop in productivity

Lizzie Gibney

Welcome back to the Nature Podcast, this week: the effects of spaceflight on the human body…

Nick Petrić Howe

…and how working from home, some of the time, could be good for employees and employers. I’m Nick Petrić Howe.

Lizzie Gibney

And I'm Lizzie Gibney.

<music>

Lizzie Gibney

As we’ve heard many times on the podcast recently, space exploration is big business right now. Launching humans into space is no longer the exclusive realm of government agencies, as companies — often backed by billionaires — get in on the act. Currently, many commercial missions are short-haul flights into orbit, helping the companies behind them test and perfect their technology ahead of future flights to the Moon and beyond.

And the people flying on these missions often aren’t specially trained astronauts, who’ve undergone years of rigorous preparation, they’re folk like you or me — although in many cases significantly wealthier.

But while there is this rapid expansion in access to space, there remains a lot that researchers don’t know about its effects on the human body. This week, Nature journals are publishing over 40 papers looking at the effects of spaceflight on health. Central to this is a wealth of data collected during SpaceX’s Inspiration4 mission, which blasted off with an all-civilian crew in 2021. During their three-day orbit, floating inside the specially designed Dragon Capsule, the crew collected a multitude of biomedical samples, which have been collated and analysed back here on Earth and are available for all to access via the new web portal.

One of the researchers behind the work is Christopher Mason from Weill Cornell Medicine in the US. To tell us more about the project and what it found, he spoke with Benjamin, who started by asking about what researchers already knew before the Inspiration4 mission, and how the project came about.

Christopher Mason

We have some knowledge of what happens to the body in space. For example, I was one of the principal investigators for the NASA twin study, which followed Scott Kelly in space for a year, and compared it to his twin on Earth. And that was the first time we really got to use the large battery of molecular biology for s pace flights. So that was about 10 years ago we began doing some of that work with NASA. But when SpaceX started to look at launching crews, we started chatting with some of their medical operations crews, and they said, this would be great to start to deploy for the civilian crews. And I said, well, we could try do everything we do everything we do for what NASA has done before for the previous missions, but we can add in all these other methods to do single cell profiling, cell-free RNA and cell-free DNA. You can see what happens in the blood of which tissues are under stress or have recently had cells that die. And so we were able to deploy a lot of the things that we do for research protocols and clinical protocols at our hospital, and then, you know, basically get them all deployed for their crew members.

Benjamin Thompson

So you had this opportunity then with these four regular folk who were the crew of SpaceX’s Inspiration4 mission, who underwent some training, but not the sort of raft of training to be an astronaut or a cosmonaut or what have you. And they went up in this mission in 2021 for a few days, quite a long way away from Earth. What was the key aims of what you wanted to do?

Christopher Mason

Yeah, there were several scientific experiments that were flying. One of the goals was really to look at, can we measure radiation impact on the crew? And they actually were getting, you know, a fair amount of radiation at high elevation. It went to 590 kilometres. So we're looking to see, could we detect that in their blood and their DNA. We also flew ultrasound experiments to look at, you know, what's the jugular vein change and does it get more stiff? We looked at the microbes on the skin and the stool. We also then looked at urine, saliva and cognitive data. And wanted to bring all that together to say, are there any things that we should worry about?

Benjamin Thompson

I mean, was it difficult to set the folk up for this? Because professional astronauts, if I can use that turn of phrase, have 1000s of hours of science training, for example.

Christopher Mason

Yes.

Benjamin Thompson

So how difficult was it to work with them and get them to collect what you needed?

Christopher Mason

It was actually easier than I would have thought. I had the same concern going into it. You know none of them had done microbiome collection, the stool collection. You know and teaching proper sterile technique. We did a viral immune assay. I had to teach them how to use that, teaching them all the protocols for collecting DNA and RNA and microbes. But they're really very trainable. So within a matter of I think a few weeks, I did a briefing we went through the protocols. They were all ready to go, and SpaceX really shepherded them through the training with relative ease it seemed like. It does help that the modern dragon capsule is mostly automated. That makes it easier on the crew, but they had to go through all the training and also the scientific training, but they did a fabulous job.

Benjamin Thompson

And so you've collected all this data then, what did you see during this three day mission? What were some of the standout findings from these participants?

Christopher Mason

Some of the big surprises were telomeres got longer, and telomeres are kind of like the ends of your shoelaces that have these little plastic bits that keep them safe and contained, and so that's on all of your chromosomes. And what we found is actually, you know, when they were in flight, the telomeres got a little longer, which was interesting, because we've seen this for other missions, but we didn't think it would necessarily happen within only a few days, but we could see that in flight in their blood, which was fascinating to see. We think now that we've studied this for several years, radiation is the big driver towards telomere length. And if you radiate cells on the Earth, you can see their telomeres will get longer. We also could see that their immune system was responding, was on high alert, so the body trying to understand, do I have an infection? Do I have any stress? We could see these, what are called cytokines, which are these little molecules in the blood that tell the immune system to get active. And then we could also see it when they landed back on Earth. We could see this spike again when they landed, including something called Interleukin-6, which is a spike that kind of restructures a lot of the vasculature and inflammation. And we now call that the return to gravity cytokine. And then a third thing, I'd say, we saw the microbe move between the crew members, and some of them became more like the spacecraft, and then became more like each other. So we could see bacteria and viruses moving around between the crew members, which was kind of fun to see that it was that fast that you could see that change.

Benjamin Thompson

I mean, I'd imagine the effects of blasting off into space is going to be quite stressful on the human body, and microgravity is, of course, a strange environment, and that would introduce changes as well, but you did compare what was happening before, during and after this three-day mission.

Christopher Mason

Yes.

Benjamin Thompson

Did these changes return to normal, or was there anything long term that you picked up on since?

Christopher Mason

The good news is that most things return to normal. We calculate, for most of the crew members, on average, about 95% of the genes that change come back to normal. Genes are, by their design, very plastic and responsive parts of our DNA and our body and our RNA, and we could see that they of course responded because the human body for these crews are in the ‘holy crap I'm in space moment’, and they respond quite quickly. But within really about a month, 95% on average return back to normal. And so what we're doing now is keeping an eye on those last 5% that seem to be perturbed and seeing if they change at all, and if there's any risk of, say, markers that are related to long-term disease, or if there's any risk for cancer long-term, which we don't think there is, so far, none of them at all are red flags. There's maybe only, you know, green flags and yellow flags are just the things we want to keep an eye on long-term. But overall, it seems to be fairly safe, at least for this crew and this mission.

Benjamin Thompson

You mentioned there that you've seen some of these things in other missions. And of course, professional astronauts can go into space for a very, very long time. The crew that you've been working with were only up there for three days. Is there any comparison you can make between the two? Any useful inferences there?

Christopher Mason

I mean, we thought they might be very different because there's a short mission versus medium, say, month or long, like yearlong missions. We weren't sure what the overlap would be going into the study, but we actually observed a lot of the same gene expression networks involving immune activation, DNA repair, UV response. So you can see that really, this radiation response was very consistent. And there's something that's called viral activation, which we can see evidence of that that the immune system, even though it seems to be on high alert, it seems to also be perturbed, because about 55/60% of astronauts have herpes virus reactivation. We didn't see herpes reactivation in this crew, the Inspiration4 crew, but we did see some of the signatures of the immune system and viral activation, viral movement in the microbial data. So we can see in some things, if you're only up for a few days, maybe you just are at the beginning stages of what would occur later.

Benjamin Thompson

And you've collected a huge amount of data, terabytes upon terabytes, I'm sure, about all these different things that you were looking at. And this forms a central part of one of the other papers, then, which is this SOMAportal.

Christopher Mason

Yeah.

Benjamin Thompson

Now this seems to be a place where others can access the information that you've gathered. Databases on astronaut biomedical data already exist. How does this one differ? And why is another one needed do you think?

Christopher Mason

So, there actually is no place you can go to download or play with or even analyse a lot of the human space-like data. So this is really the first release of a human medical and genomic portal where you can look at data from the twin study, the Inspiration4 mission, other mice studies that have flown in space. So we aggregated the totality of all human and mouse data that exists, that's publicly accessible and that we've made into one portal. So we mainly wanted to make this for the community, because a lot of people would ask us, hey, I think this gene might be important in space, can you let me know if you saw it in your data? Or I think this microbe is important, or I have a drug I want to try can you tell me what am I target? So this makes it a nice nucleation point for testing a hypothesis before you try a new mission into space.

Benjamin Thompson

And this is publicly available?

Christopher Mason

Yes, it's launching with a launch of the papers. The other thing that's important is that we had to consent the crew, so we had to make sure that they understand that they're contributing to research and that the data will be publicly available. But they've been extraordinarily excited and giving to say, if my data can help future spaceflight, then please share it with the world. But that's not always the case. We have almost no Russian spaceflight data. We have some data from JAXA, the Japanese Space Agency, on the data portal, some data from ESA, the European Space Agency. So we really want to make this an anchor point where any data from anyone human or animal that's been flown, we can actually query it, learn from it, and integrate it.

Benjamin Thompson

A lot of this is based upon the data you've collected from these four folk, and detractors might say that they had a limited amount of time in space, there's not many of them, there was variation seen between them as well, and there weren't mirror controls back on Earth at the time. Are these results significant do you think? What use does this data have?

Christopher Mason

So we're analysing as many of the samples as we can from astronauts, but it's notoriously low sample size, kind of science, because there literally aren't other samples to get. And so what you lack in sample size you can make up for in frequency of sampling and then the breadth of the diagnostic and molecular research that we do. So what we do is, even though it's only four people that went on Inspiration4, and we only have, you know, two twins in the twin study, but what we've done is link it to other missions, like axiom astronauts or other NASA astronauts we're also still profiling. So, you know, all told, in the database, we have about 64 astronauts with some data. So on that level, you know, of all who have been in space, we've got about 10% of their data on these immune system measures. And so what we do is, for any individual small crew, like four people at a time, we compare them to the bank of 64 or to the list of changes that happened in the twin studies. So we build a portal so you can actually compare between the missions. And so we can start to have at least the beginning of some statistical power to look at differences between groups, between missions, ages and sexes, and it's still very low relative to studies where we have millions of subjects. Of course, that's ideal, but this really gives the foundation for what will eventually be the case where we have 1000s and 10s of 1000s of people living and working in space.

Lizzie Gibney

Christopher Mason talking with Benjamin Thompson there. Head over to the show notes where you can find links to the papers associated with the project.

Nick Petrić Howe

Coming up, does working from home damage productivity? Maybe not, at least if you’re in the office some of the time. More on that later, right now it’s time for the Research Highlights, with Dan Fox.

<music>

Dan Fox

For much of 2019, Europe, the Middle East and Africa were awash with painted lady butterflies. Now, researchers think they know why. Painted lady butterflies follow a complex, multigenerational migration, the entirety of which no one butterfly ever completes. This migration pattern made explaining the 2019 event particularly tricky. So, to identify the source of the butterfly outbreak, researchers analysed pollen grains transported by 264 butterflies collected from swarms across ten countries. DNA analysis of the pollen allowed the team to infer the insects’ origin. Community-science data confirmed the story told by the pollen: the butterfly outbreak probably originated in northern Arabia and the Middle East, where vegetation was unusually abundant in late 2018 to early 2019. This initial boost cascaded into an intercontinental event spanning nine months, with the population boom spreading to Eastern and then Northern Europe, by which time, the swarm probably represented a second or third generation before finally arriving in Western Europe in June, July and August 2019. If this butterfly research has caused a typhoon of interest, you can read the paper in full in Current Biology.

<music>

Dan Fox

Along a stretch of the Orinoco River that cuts between Colombia and Venezuela there are dozens of supersized rock engravings, including a giant snake 42 metres long. Now researchers might know what these artworks mean. A team used drones to help map almost 160 rock-art sites along the Orinoco they found 115 engravings across more than a dozen locations that measure more than four metres in one or more dimensions. Some of the monumental images depicted snakes, which feature prominently in the myths and beliefs of local, indigenous communities today. The giant snake engravings are in an area known to be a hotspot for ancient communities on the move, and would have been visible to people from hundreds of metres away suggesting they were used to mark boundaries between the territories of different groups. You can find that research in Antiquity.

<music>

Nick Petrić Howe

Working from home is certainly a controversial topic, but evidence is mounting that a hybrid approach — some time in the office and some at home — may be beneficial for both employees and employers.

Nick Bloom

So in this one study, we see quit rates drop by a third. Other studies and other measures and surveys reported employees value hybrid about the same as almost a 10% pay increase. So in a sense, it's like, what's not to like?

Nick Petrić Howe

This is Nick Bloom, a Stanford researcher who’s been looking at the effects of working from home for over 20 years. He and his colleagues have come out with a new study trying to identify the pros and cons of a hybrid working approach. It comes at a good time too as social distancing fades into memory and companies reconsider their policies, there are heated debates about whether working remotely is a godsend or the work of the devil. Many of these debates are based on anecdotes and general ‘feelings’ about what is the best approach. So, what does the science actually say?

Nick Bloom

There is just huge dispute over impacts of work from home on just about everything. I think the stuff that's the safest bet is that people like it. They typically don't want to work from home every day. In terms of the impact on productivity, it looks like fully remote, so if you never come into the office or workplace, is probably negative. But even there, there are some studies claiming it's positive.

Nick Petrić Howe

Many past studies have looked at working fully remotely for jobs like call centre staff, helpdesk workers and data entry roles. It’s here where the negative effects have been found, but those kinds of roles don’t really represent all the jobs that are now done, at least partly remotely. So, to cut through all the anecdote, Nick B. and team trialled a very scientific approach — a randomised controlled trial. At a tech company in China, Trip.com, 1612 people were randomly assigned to either work from home for two days a week, or work in the office five days a week.

Nick Bloom

So we ran the study for six months, we then collected a further 24 months of data after that. So what do we find? So fact one: was there was just no impact on performance. It wasn't positive, but it wasn't negative either. We have pretty good data on that, so we have performance reviews, you know, what the grades they get. And in fact, all the text we can do natural language processing on, we have promotions, and for about half of them, we have lines of code written and product shipped. And across all of those variables, you find nothing. And this is a pretty big sample. You're looking at 1600 employees, so standard errors are kind of tight. So, you know, effectively, it was a big fat zero. And it's what, when you talk to people on the ground, they said the same thing, you know, sure, working from home a couple of days a week, you're a little bit less connected. Maybe it's a little bit harder to mentor and train. On the other hand, you save a lot of time. It's quieter, it's less stressful. Net net doesn't seem to have an overall impact. The other key finding was employees really liked it. So on surveys, they were really positive.

Nick Petrić Howe

Employees rated their work-life balance, work satisfaction and life satisfaction significantly higher when working hybrid compared to those who were in the office every day. For the employers, to start there was no reduction in productivity, which has been a major criticism of home-working, but then there was another effect that they were very interested in.

Nick Bloom

Maybe the hardest data is quit rates fell by 35%. And, for the company, they looked at it overall, and they were like wildly positive. So their view was ‘look, there's no effect on performance at all. We maybe save a bit of space in the long run, but critically, we're reducing quit rates. And each person that quits cost us about $20,000 because you get to go out and rehire somebody new. You gotta train them up. You gotta take manager time to do that.’ They said, ‘it's just expensive to have turnover.’ And so Trip, at the end of the study, just rolled this out to the entire firm, and in fact, now a bunch of other companies, they were based in Shanghai, a bunch of other companies copied the same policy.

Nick Petrić Howe

Reducing quit rates by over a third would generate millions of dollars in savings, at least for this company. The reduction in quitting seemed to affect some more than others too. People with longer commutes and non-managers became less likely to quit than others when switching to hybrid. The quit rates for female employees also saw a 54% reduction. Despite this though, female employees were far less likely to volunteer to be part of this study in the first place.

Nick Bloom

So an interesting twist in the way the experiment was run. As initially, they asked everyone who'd want to volunteer, and only about a third of people volunteered. In the end, they just rolled it out to all 1600 people in both divisions, but we still know who opted to volunteer at the beginning. What you see is, firstly, actually even non volunteers took it up, so as soon as they're actually allowed to work from home then they took it up. So it looks like a lot of people didn't volunteer, not because they didn't want to work from home, but they're worried about what it might signal. So when you interview folks, they're like, ‘well, if I volunteered to work from home, does my boss think I'm a slacker? And, you know, does he or she not promote me as much?’ Second finding was women saw the biggest gap between volunteering and actual take up. And, you know, maybe they're the most nervous about volunteering. Maybe they're the ones that most at risk of being perceived as, I don't know, less engaged. The big ‘so what’ for us was actually having a companywide policy is really important in the sense that a lot of people who would like to work from home two days a week, and if you position it as, you know, it's optional, but whisper quietly, but, you know, highfliers will come in. You know, it really doesn't work out that way. What you really need is like, managers to say it's the norm. We expect you to actually work from home on these couple of days, as much as we expect you to come in on the days that there are office days.

Nick Petrić Howe

Those expectations of which days people come in could be important to how well this works, as that part particularly interested Marina Schröder, an economist who studies creativity and home working and wasn’t associated with this study.

Marina Schröder

The intervention they did made sure that there’s still three days a week where everyone meets in the office. And I think that’s a very promising approach.

Nick Petrić Howe

Marina’s previous work has shown that communicating with colleagues through things like chat software can lead to less innovation when compared with face-to-face conversation. She does caution that these results do vary based on the task, but it adds to other evidence, like a Nature paper from two years ago that showed that video calls could lead to less ideas generated in brainstorming sessions. Having people come in on specific days though, could help mitigate these effects.

Marina Schröder

I think this paper suggests that hybrid may be a good combination. But for practitioners, it's important to look at the type of hybrid they implemented, which is really having a couple of days where everyone is in the office and a couple of days where people can work from home. I'm saying this because for practitioners, once we're talking more about working from home or hybrid work arrangements, then they're also often thinking of reducing rented office space, which may not work in the context of the hybrid working arrangement implemented in the study here.

Nick Petrić Howe

In the future Marina would be interested in seeing what the effect would be if employees were able to choose which days they worked from home. If people didn’t overlap, would you see the benefits shown in this study without the drop in productivity? Overall, though, Marina was very impressed with this study. It’s often difficult to do such large studies where companies randomly select employees to do one approach or another.

Marina Schröder

So getting a clear control group in the context of working from home is very difficult. And the fact that the authors could overcome these challenges and actually have a clean control group, and the treatment group of participants were allowed to work from home Wednesdays and Fridays, that provides very valuable insights.

Nick Petrić Howe

One other thing to consider about this study is how generalisable it is. A trial like this one is great to get fine-grained information, but it’s just about one particular place at one particular time. Nick B., though, thinks that there’s a lot of evidence to recommend hybrid working.

Nick Bloom

The results here look very similar from survey and more kind of epidemiological type data looking at us and Northern Europe. So just to be clear, in the US, we have very good data on how many days companies, on average, let their employees work from home. What we see is companies that let their employees work from home two, three days a week are growing faster since the beginning of the pandemic. They have higher stock market returns, they're performing better. There's a famous study looking at RTOS return to offices. It's called the Pittsburgh study. What it finds is companies that announce return to offices typically have been doing particularly badly and have falling stock prices. And announcing a return to office does not appear to, you know, turn that around. So the evidence at the other extreme, just looking at 1000s of companies, is also that hybrid, at least, is associated with a stronger company performance. I mean, you know, step back even further US, northern Europe's economic performance post pandemic has actually been reasonably good given all the, you know, headwinds and, you know, people are kind of amazed about how fast the recovery has been. I think one factor driving this is actually hybrid has been pretty positive.

Nick Petrić Howe

As companies across the world look critically at their working from home policies, there is definitely more to understand and more studies will be needed. But from the perspective of the science that we do have, there seems to be a growing evidence pool that hybrid, at least, may let you have cake and eat it too.

Nick Bloom

If you can let people work from home two days a week and their performance doesn't suffer, and they're much less likely to quit and they're happier, then it just seems all good.

Nick Petrić Howe

That was Nick Bloom, from Stanford University in the US. You also heard from Marina Schröder, from the University of Hannover, in Germany. For more on that story, check out the links in the show notes.

Lizzie Gibney

Finally on the show, it’s time for the Briefing Chat, where we discuss a couple of articles from the Nature Briefing. Nick, what have you been reading this week?

Nick Petrić Howe

Well, this week I've been reading a story in Nature, which I think taps into many worries researchers have, because it's all about funding. But in this particular case, it's a story about the funding of a future Super Collider by CERN.

Lizzie Gibney

Which I'm going to assume, well, I know, because I've written about this before, is very, very, very expensive.

Nick Petrić Howe

Yeah, it certainly is. It's estimated to cost around 17 billion US dollars, so certainly not a small experiment by any means. So what I'm talking about here is what is known as the Future Circular Collider and this is a collider that's going to be a 91 kilometre circular particle accelerator, which is hoping to probe the Higgs boson further. So the goal of at least the first stage of the future circular collider is to mass produce the Higgs boson — so just make loads of them. And this could help us understand things like dark matter, cosmic inflation, and you know, some of the big questions in physics.

Lizzie Gibney

So this is CERN's plan. So there are a few different organisations around the world that are kind of mulling future big colliders, which all would probe the Higgs as well. It's definitely a big focus of exciting physics in the future. And in this story, there is a little bit of a question mark about whether CERN could actually fund the collider that it's planning.

Nick Petrić Howe

Yeah, that's right. So one of the big funders is the government of Germany, and they have said that the current project is unaffordable. So Germany gives CERN 20% of their current budget, but this would be a big increase, because you'd need the extra 17 billion to create the Super Collider. So they've said it's unaffordable, which obviously raises a lot of questions. So this is an interesting story as well, because it's a bit of a scoop. It's actually come out of a workshop for particle physicists in Germany, and they found out then that it looks like Germany may not be willing to fund the new Future Circular Collider. We don't know yet how some of the other big funders, like the United Kingdom and France, what they think about the feasibility of this, but in the first instance, Germany has said that it's unaffordable.

Lizzie Gibney

And did they say what was behind that? If they don't think that this is a good idea anymore, it’s just simply too much money?

Nick Petrić Howe

So one of the things that a spokesperson for the Federal Ministry of Education and Research of Germany has said is ‘that the preliminary cost estimates for this collider are subject to a lot of uncertainties, and the effects of those are still unknown’. They think that the financing plan is a bit vague and it will require a lot of commitment from partners like Germany, so at the moment, they can't support the funding for the project. I mean, I'm going to assume that in the backdrop of everything that's happened in the world over the past few years, money may be tight for many governments. And you know, $17 billion for a Super Collider may not be on the cards for them at the moment, but we can only speculate as to exactly what the German government are thinking. What we do know is that CERN has done a feasibility study on this first stage of this Future Circular Collider, and that's where the numbers have come from that Germany have said, like, ‘whoa, this is maybe a bit too much money’, but it's maybe not the end of the story. There are other ways that this could be done. Rather than building a big circular collider, you could build a linear one that could be cheaper and may save energy. So that's a possible way forward. And CERN, for their part as well, have said that this new collider will mainly be built from their main budget. But others have questioned this, because a lot of that budget is obviously used for their current operations and an upgrade they’re planning as well. So big question marks really about what's going to happen here.

Lizzie Gibney

So from reporting I've done in the past, I know that to get any of these enormous facilities built, you need a lot of political wrangling, and so I'm sure behind the scenes, there'll be a lot of some leadership talking to people high up in all different countries. And so I wouldn't write it off yet. So I think the LHC costs something like 10 or 8 billion euros – so this is more than that. But these things are always hugely expensive and especially in this case, you know, they've got a tunnel out a ring, that as you say is 91 kilometres long under the ground. So that, in itself, just costs an enormous amount. But once it's built, it becomes this hub of, you know, the entire world's particle physics community. So this is the way that it's tended to work in the past few decades, is for there just to be one massive facility like this that everybody comes to somewhere in the world, but there is always a lot of jostling to figure out where the next one is going to be.

Nick Petrić Howe

No exactly. And there are others that are planned, as you mentioned, as well, and that may come into a little bit of the politicking too, because there is one of these big supercolliders planned in China. So it could be that the Europeans don't necessarily want to lose their edge. So maybe that's something that the people at CERN will be saying to governments of Europe. But at the moment, as I say, just question marks.

Lizzie Gibney

And I think there's also a question mark over whether any new massive facility like this will be built. Within physics right now, there is a sense that we don't really know where it's going. We know that there are a lot of holes in what we know. There are things that are not explained, like dark matter, like why there's matter everywhere and very little antimatter. So we know that our current theories aren't right, but we don't really know where to look or how to look for the answer, because we don't know what the answer is. We have no good leads when it comes from theory. So I think more than ever there are also people questioning whether just this strategy of keeping building bigger and bigger facilities colliding particles, you know, greater energies, or even with, you know, higher precision, if that's actually gonna be the way that we answer these questions, that's more uncertain, I think, than ever before.

Nick Petrić Howe

No, no for sure. So we'll be keeping an eye on this and other super colliders as we go forward. But for now, I'm interested in what your story is this week. Lizzie, what have you got for us on the Briefing Chat?

Lizzie Gibney

Yes. So this comes from our Robotics and AI briefing. It's a story that was in the Associated Press, and it's on when we will run out of data to feed these large language models, which everyone will have heard of. They're the kind of models behind chatbots like ChatGPT, and they've been getting bigger and bigger and bigger, and that's the way they recently have been getting better. So that's what the developers do, is they feed in more and more data, and these models get better and better. But this study that's reported on in this article, which comes from a nonprofit called Epoch AI, they've looked at when that data, that human written text, that the LLM’s — these large language models — can train on, when that will run out. And this report has said that that's going to be probably between 2026 and 2032 so that's not all that far away. And it flags that there may be, you know, really big challenges in maintaining this pace of progress that we've seen recently if this resource runs dry.

Nick Petrić Howe

There has been a few different estimates, I think, of when this data might run out. I think 2026, was one year that was suggested. So what are the sort of assumptions and things they're considering to come up with this number?

Lizzie Gibney

They're looking a little bit more accurately at how the models work now, because that's shifted a little bit the kinds of data that they can input and the efficiency of the models. So that's why this has moved to, you know, 2026 to 2032, but it is quite difficult, I think, because it's quite murky what these models are trained on in the first place. And what's happening now is that firms like OpenAI, like Google, who develop these models, they're trying to buy up more of this data that's out there. So they want the high quality text they want, you know, to buy, for instance, from different news media outlets and to do deals with Reddit and, of course, Meta, the parent company of Facebook, has also recently changed its policies around what it can do with people's data. But longer term, this study points out that all of that will still run dry. And there are a few options after that. So you know, do they tap private data? Do we somehow find a way for people's emails and text messages to end up feeding these ever-hungry models, or do developers try and use synthetic data, which is where they actually use models to generate more text, and then they train on that text. So, that gives them unending amounts of– of data. But there's a lot of concern that it's just, you know, you lose a lot of the information. You also maybe further in-bake some of the biases by doing that. And one person in the story is quoted as saying, ‘it's like photocopying a photocopy over and over again’. You're losing a lot of the quality by doing that.

Nick Petrić Howe

I almost had the impression that these companies had already kind of run dry. There was some reporting recently that many of these companies have started to go into sort of legal grey areas, scraping YouTube and other resources to try and, you know, keep training these ever-hungry models.

Lizzie Gibney

I think that's a bit of a legacy thing. I think when the models were first being developed. It was perhaps a grey area, and they also just didn't tell anybody what they were using. And so there were questions over whether, you know things like YouTube, is that– it's publicly available, but is it public domain? Is it actually able to be used by these models? Especially now that they have the potential to be very profitable. So I think we don't know exactly, and that's one of the problems that I also written about in different stories about these models, is that we don't quite know what's in them. But that is something that people are trying to unravel now, and that's where some of these numbers come out, you are looking at what's in them at the moment and how much there is left to gobble up.

Nick Petrić Howe

And I wonder as well if this is going to start becoming a case of diminishing returns. I mean, GPT 4 and its ilk are already very fluent in terms of text, like, how much better could they get?

Lizzie Gibney

Well, so far, scaling has worked very well in that you just put more and more data in, and as long as it's good quality text, you tend to get better and better outputs. As long as you keep scaling the compute as well, of course, because the models themselves get very, very complex. But there is a sense that maybe that is not going to last forever, that kind of scaling law. And there are also issues with using that amount of compute, you know, it's very, very energy intensive. So in parallel, there have already been lots of efforts going on to try and create models that don't need such a large amount of input data, and that can improve by being more efficient, by just being designed from scratch in very different ways. Or perhaps that don't try and be so general, that try and just target more specific tasks, and they get really, really good in their specific areas, but don't need that, you know, entire world's worth of information to function. So this is perhaps more of the route that they're going to have to go down, because, yeah, it's got to end somewhere,

Nick Petrić Howe

Indeed. Well, I think that's also a good ending for the Briefing Chat this week. Thanks Lizzie. And listeners for more on those stories and for where you can sign up to the Nature Briefing to get more like them, check out the show notes for some links.

Lizzie Gibney

And that’s all for this week, as always keep in touch with us on X, we’re @NaturePodcast, or send an email to podcast@nature.com.

Nick Petrić Howe

And if you’ve enjoyed listening, don’t forget you can always drop us a review wherever you get your podcasts. I’m Nick Petrić Howe.

Lizzie Gibney

And I'm Lizzie Gibney. Thanks for listening.