Stephen Wilson 0:06 Welcome to the Language Neuroscience Podcast. I'm Stephen Wilson and I'm an aphasia researcher at the University of Queensland in Brisbane, Australia. This is Episode 29, recorded on the 29th of March 2024. It's been a while since the last episode, I've gotten way too busy, but I've really missed recording episodes. So I'm getting back to it now and I've got some great guests lined up. Before we get started. I also want to say it was great to meet heaps of people who listened to the podcast at last year's SNL conference in Marseille. This year, as you probably know, SNL is coming to Australia for the first time, to the Brisbane Convention Center, which is like ten minutes from where I live. I really hope that many of you will make the trip for the conference. I think it's going to be a very special one. And you'll love Brisbane. It's a beautiful city. You can catch ferries up and down the river, there’s beaches to the north and south of the city, subtropical bushland throughout. I can give you insider tips on where you can see koalas in the wild, guaranteed. Or if you want to pet one, there's places you can go for that too. My favorite animals in Australia though, are the birds. You'll meet kookaburras, sulphur-crested cockatoos, bin chickens and many more. We really do have the best birds in the world and they all have their own unique personalities. Wouldn't you want to wake up to sounds like this ? (Kookaburras singing a beautiful mix of cackling 'laughter', chuckles and hoots.) Those are kookaburras. They live all around my house and they usually get going with that at about four in the morning. It's better than any cup of coffee to get your day started. Well have a think about it. I hope that many of you will come to Australia. Okay, let's get to our episode. My guest today is Dorothy Bishop, Emeritus Professor of Developmental Neuropsychology at the University of Oxford. Dorothy is without question one of the most outstanding language researchers in the world. She is a world leader in the study of children with language impairments. And I had a lot of fun delving into just a few of the hundreds of papers she's written over the course of her career. Her papers are beautifully written, methodologically rigorous and she's always asking the big questions. All right, let's get to it. Hi, Dorothy. How are you? Dorothy Bishop 2:07 I'm good, thanks. Good to meet you. Stephen Wilson 2:09 Yes. Good to meet you, too. I think we've chatted a few times online, but I've never actually met you in person or on Zoom. Dorothy Bishop 2:15 No, no. Stephen Wilson 2:16 So it's morning time for you over there in England. Dorothy Bishop 2:18 Yeah. Stephen Wilson 2:19 Where are you? Where are you now? Dorothy Bishop 2:21 Oxford. Stephen Wilson 2:22 Uh huh. Okay, I always like to start by asking people how they got into the kind of work that they ended up doing. And for you, I've, you know, early in your career, I think you settled on the questions that have driven your whole career. But I'm really interested in how you got, you know, to that, to those questions. So, did you have an interest in language when you were a kid? Dorothy Bishop 2:45 Not particularly. I mean, I enjoyed learning foreign languages. But it wasn't something I thought I would be going on to study, I think I came at it more, that language became the bit of brain processing that I was focusing on. So, my early interests were very much in science. And I enjoyed most scientific subjects, particularly biology. If I hadn't been gone into psychology, the, I was going to do zoology. And I'm still always, you know, just fascinated by animals and so on. But I think I was coming at it more, that I was interested in biology, but I was also increasingly interested in people and why people are different from each other, why there's such individual differences in humans. And it was, I mean, most of the decisions in my career are things that happen by chance. And at school, you could, it was, it was in the days when, if you wanted to go to Oxford or Cambridge, you had to take a special entrance exam before your final school exams. That's all changed now. But in those days, that was how it worked. And my school didn't really routinely sort of get people going through that. It's just a regular state school. But if you said you're interested, they would try and help sort of thing and I thought, why not give it a go? And then you had to sort of decide what subjects did you want to take, and I was very uncertain, you know which direction I wanted to go. The other thing I did like was maths, but I think I was, I felt that would be a bit too, you know, not appealing to my more social side. So, I ended up applying to do initially, at Oxford, I applied to do physiology and philosophy, which is a combination that nobody does. And this was typical, I mean, I was always a bit sort of trying to be counter, countercultural or whatever. But I knew I was interested in physiology. And I also thought, I had no idea what philosophy was. I was ridiculously ignorant. I had, I had sort of vague ideas of sitting around thinking great thoughts. And the Oxford degree had this three P's, PPP, which was philosophy, psychology and physiology. And the idea was most people took two, one of which was psychology. And I was opposed to psychology because I had read some Freud, and I thought the man was an idiot. Stephen Wilson 5:30 Okay. (Laughter) Dorothy Bishop 5:31 I didn't want to do that stuff struck me as very unscientific, and hand wavy. And so I applied for philosophy and physiology. And then I got interviewed and that was also rather interesting, because I mean, when I did, I did, I did an entrance exam also for Cambridge. And I just flunked that completely. Stephen Wilson 5:54 Okay. Dorothy Bishop 5:55 I had, what they didn't tell you at my school, and I think what they perhaps didn't know, was that these entrance exams were not designed to test what you've been taught. They were designed to test whether you could think, and there were questions about things I knew nothing whatsoever about. Stephen Wilson 6:09 Okay. Dorothy Bishop 6:09 So, I was just thinking I know nothing whatsoever about this, I'm obviously not suited for this thing. And when it, that was the Cambridge one, which was just hopeless, and the Oxford one, I was asked these questions, I thought well I haven't been taught this, but I might as well write something. And I remember I sort of got in because I wrote an essay about why there's, trees, why there are some trees that have sex and some trees that don't have sex differentiation. And you know that the question is really, I mean, I can't even remember about it now. But there's male and female trees, and then there's trees that are both male and female. Why should it be, you know, and I sort of just thought it through and wrote what I thought and I guess, I don't doubt it was very profound, but I guess it showed that, in that situation, I could try to hash something out, which is what they wanted. So, my essay on individual, you see this again, individual differences, why is everyone different. And my, so I got in to get as far as an interview. And then when I got interviewed, my philosophy interview was a disaster, as you can imagine, they sort of sat there saying things like, well, how do you know that other people feel pain? You know, how can we how can you tell that? And I sort of said, well, you know, and I think if I was asked that question, I would sort of get a whole load of people. And I'd ask them if they're feeling pain, and then I take some physiological measurements and try and find some sort of correlate, but they looked at me and they said, I think you should do Psychology. (Laughter) Stephen Wilson 7:37 Did they tell you that they had come a long way since Freud and that like, you know, you didn't have to worry? Dorothy Bishop 7:41 Yeah, well, in fact that when I got this interview with a very nice psychologists Til Cohen, who, who just sort of, basically, yeah, I mean, she said, I saw the word as I walked in, that was experimental psychology. I thought, our experimental this sounds more like me. And I realized that, you know, she did indeed sort of make it clear the sorts of things that they studied, and I realized this was very much exactly the sort of thing that interested me. And so in fact, I came to Oxford, and did philosophy did physiology and psychology for two terms. And then I switched to just straight psychology because I was enjoying the physiology so much, which involved a lot of animal experimentation, which I wasn't opposed to, in principle, but I just found it made me feel, you know, I think now, I do think in principle, probably students were given rather too many animals to dissect, and so on. I mean, if you're going to go on or something, that's fine. But in those days, you know, we were all chopping up frogs and I remember some very unfortunate tortoises that had their shell removed and so on, and I didn't like it. I mean, I, so I just switched to straight psychology. Stephen Wilson 7:41 Uh huh. And you did go into removing the shells, the shells of little kids but like metaphoric shells. (Laughter) And, and so, okay, so that's how you got into Psych. So you kind of heading to the field of language disorders through Psych. Well, you mentioned like language was the window. Dorothy Bishop 9:20 The person, the person who really got me interested in language actually was in the course of that undergraduate degree, there was a whole series. I mean, again, these are things that you wouldn't get these days of very eminent visiting academic coming for a whole term to do a lecture series to undergrads. And it was Hans-Lucas Teuber, talking about you know, impact on the brain of people who had gunshot wounds, focal lesions, and he was fascinating. I mean, I was just absolutely hooked at that point on neuropsychology. And so, I felt that that was the area I wanted to go into. It still wasn't specific to language and perhaps it never has been I mean, I feel I'm more generally interested in neuropsychology, but I felt at that point that neuropsychology was a topic I just was, again, it appealed to this underlying interest I had in why people differ from each other and you know how much it was related to whether the brain was wired up one way or another way. I did an undergraduate practical that has left a very long mark on me with a guy called Tony Buffery, which was involved dichotic listening. And we just did dichotic listening and show that, you know, one side, one, you had a left ear advantage of most people. I'm left handed, so I was very interested in the idea that maybe I was different. Stephen Wilson 10:43 Are you left lateralized for language? Dorothy Bishop 10:45 Yeah. Boring. (Laughter) Yeah. So, but I that that got me, you know, again, I just, perhaps that was an early sign of interest. And I did do I think I did my own project on dichotic listening as a consequence of finding that just a very intriguing observation. Stephen Wilson 11:05 Yeah. No, I looked at your, you know, I looked back at your old papers, and I can see that you had the lateralization bug very early. Right? Dorothy Bishop 11:13 Yes. Stephen Wilson 11:13 That was like, that's kind of one thing that I really noticed about you. It was like all the strands of your work, were there like really early? I feel. Dorothy Bishop 11:21 Yeah. That's true. Yeah. Stephen Wilson 11:25 So, how, yeah, so I think so how did you get from, how did you get from there to, I mean, I guess like, the, the first like paper that you did that really made a big impact on the Test for Reception of Grammar. How did you get, how did that come about? Dorothy Bishop 11:43 Right. So, I went then, after my undergraduate, I went and did, I did a clinical psychology Master's. And these, you can't do that these days. Now it's a doctorate. But in those days, it was a two year master's degree. And I did it in London at the Institute of Psychiatry. And I thought I might want, I still didn't know really what I wanted to do, but I thought I might want to be a clinical psychologist, but probably specializing, specializing in neuropsychology. But we did, you didn't do much neuropsychology. And it was, the little bit that we did was not well done. I mean, it was quite a disappointment after Teuber, I was really expecting, you know, we'd now get whoa! And in fact, I spent most of my time doing behavior therapy. Very little with children, too, surprisingly, given where I ended up. But it was mostly it was in the heyday of the development of behavior therapy prior to, I mean cognitive behavior therapy was sort of on the distant horizon. But this was the early sort of just behavior therapy, get away from psychoanalysis. So that appealed, get into, you know, sort of really just using behavioral principles to treat people's phobias and obsessions and things. So I did two years of that, but I really felt I'm not really a natural clinician. I kept thinking, you know, I want to know more about, I didn't feel we were in a state to treat things, I didn't feel we understood enough. And I was very curious about the phenomenon. And I did really want to do more neuropsychology. So after that, I also wanted to move to Oxford because I wanted to be with my husband. I mean, we weren't married but we wanted to be together. He was in Oxford. So, I found that there was a chance to get a job with Freda Newcombe, who had a little neuropsychology unit in the Radcliffe infirmary in Oxford, and her, she did a lot with gunshot wound patients. She had made, she'd done her own PhD on that same group of gunshot wound patients was very interested in reading in particular. I did some of the very early work on deep dyslexia and so on. So, I got a job with her thinking, and it was possible to then register for a doctorate, but I was hoping to work on aphasia, I still wanted to work on acquired aphasia. That proved really difficult. It was difficult to get access to patients, there were all sorts of political issues. And freda, I think very wisely, was steering me towards children, which is, she was getting quite a few referrals of children with weird language. And she didn't really that wasn't her thing. And she didn't know what to do. And but she felt that this was something that really was a topic that somebody somewhere should get their teeth into. And I think it was a good advice, although I was not a sort of natural developmental psychologist and hadn't done much developmental psychology in my degree. (Laughter) So… Stephen Wilson 14:42 Perhaps you were a natural in retrospect? (Laughter) Dorothy Bishop 14:45 I don't know. Well, that it was it was a wonderful opportunity because it was at a time in Britain where they had just opened a school, a residential school for children with language problems in up in the north of England in Mansfield, near Mansfield. And the principal was a woman called Sandhya Naidoo who had done her PhD on dyslexia, but had been appointed as the principal of this school to run a school on children with language problems. And we were invited to visit and see some of the kids. And then there were two other schools at the time in the UK, that were really helpful and encouraged researchers, there was More House school which is still going strong in the south of England, and was founded by a medic, who was a pediatric neurologist, who was, who had seen some children with, with this weird acquired epileptic aphasia, and he'd founded this school, and they had some of those kids there, as well as children with other sorts of, you know, language problems of unknown origin. And then there was a third school on the south coast, John Horniman school, and all of these were schools where you had your sort of, you know, sitting target of all these kids with problems, you know, with nobody studying them, not much known about them. So, it was a great opportunity. And one of, the test resection of grammar came about really just because I wanted to, but the question that I was most interested in was, whether you can have a pure expressive language problem in a kid with normal comprehension, and vice versa whether you ever had kids with comprehension problems, but I think unlikely that option would occur, but I want you to evaluate their, particularly their understanding of you know, how far they could understand the meaning conveyed by grammatical contrast. And there just wasn't much out there that was suitable. There were a few standardized tests available. But they were terribly confounded, you know, that they really hadn't teased apart the different factors, the things that a kid could respond to. And I was trying to develop something where you could be pretty confident that this was the characteristic that you'd had enough items. And you'd have various contrasts built in, that you could really tease apart with, if they were having difficulties, what it was that was giving them difficulties. And so I developed this test for my PhD. And then subsequently, it seemed worth developing it to sort of get it properly standardized, and everything and make it more widely available. Stephen Wilson 17:31 Yeah. And it's become very, very widely used test still to this day. And many other tests, I think are based on it. I mean, kind of having I mean, just in case people aren't familiar with it, it's, you know, an a picture matching task with I think, I think there are four choices on all the items. Is that right? Dorothy Bishop 17:49 Multiple choices. Yeah four choices. Stephen Wilson 17:50 Yeah, I mean, it's essentially, you know, you can't get them right, unless you have a functioning grammatical system. And nowadays, this is, you know, very standard, and there's probably dozens of tests would have that basic structure. But that's one of the really early ones. Dorothy Bishop 18:04 Yeah, I know, there was a few that had that tried to get that idea. But they, I think you have to have foils that are not grammatical items, particularly early on in this test. So because you can have kids who just get everything wrong, because they terribly inattentive or because they don't understand the vocabulary. And so the idea was to try and design it so that if you couldn't stop kids making mistakes for all sorts of reasons, but you could then diagnose from the pattern of results, you know, were they selectively having problems with the grammatical contrast? Or were they just sort of not very good at taking chances or attending and so on? Stephen Wilson 18:43 Yeah. And what did you find with that early work? Did you find that grammatical deficits could be selective? Or is it always going along with other things? Dorothy Bishop 18:52 I think they're not. Yeah, that's a good question. Actually, I don't know if they're totally selected. But certainly, the grammatical items were much more likely to pick kids up than others so that they you did see disproportionate problems. Let's put it that way. I mean, I'm perhaps too far to say selective. The other thing was it picked up that a lot of kids who people had thought her purely expressive problems, did have quite subtle grammatical problems when you tested them this way. So everyday life, they would be getting away, okay, by filling in the gaps. And of course, in you know, in real life conversations, and so on, you don't need all the time to be aware of every grammatical nuance. You can use all sorts of other cues. So these kids were sort of getting by, but if you sort of pinpointed them, and put them on the spot so that they had to respond to these items, then you could show that they were impaired. So, I came away with the idea that you didn't see pure, expressive grammatical problems in kids without a receptive component. And the receptive part might be more, you know, more hidden, but if you've tested it this way you could you could show it. Stephen Wilson 20:01 Yeah. And again, I think that's one of those themes that you've just followed up extensively in your career, this idea of trying to understand all the different strands of speech and language disorders and how they relate to one another. I just think that's kind of where I'm through a lot of your work. Dorothy Bishop 20:17 Because the other thing that was interesting, was that you get you go to these schools, and it was a wonderful opportunity, because the staff that these schools were mostly top notch, they were Speech and Language Therapists, they were special education teachers, they really knew that stuff. And they knew these kids, and they would sit in the, you know, in between while I was seeing some kids, they would sit and chat, and they would tell you what they thought about the children. And so you'd get these kids and they'd say, Well, this child's got really bad comprehension. And I'd go and test them on TROG and they were fine. And so I think, initially, I’d think they don't know what they're talking about, and then I realized, there are other comprehension problems. So that and there you do, you could see some in some kids, this real dissociation between, you know, the kids who are fine on the grammatical bits, but these kids were the kids that these days would get a diagnosis of autism, because they will make their comprehension was what some of the schools recall, high level difficulties. There were problems with inference. There were problems with, you know, sort of understanding what you meant, rather than what you said, and some of these kids, also, when I first met them, I thought, why are they in this school for children with language problems, because they tended to be the kids that also could speak in long, complex sentences, then you'd listen to what they were saying and it was strange, you know, so that these kids for whom the problem seemed to definitely not be with language structure, but much more with use pragmatics language function. Again, I'm making it sound like a clear binary divide. And I kept most of my career, I kept trying to find ways to make these clear binary divides. Before I gave up, I realized that most children, it's a bit of this and a bit of that, but you could see individual cases, in whom this was a very sort of just clear dissociation between the kids with the problems with the language structure aspects, and particularly grammatical contrast. And those who had not got that problem, but who still had communication problems. Stephen Wilson 22:12 Yeah. Uh huh. So, can we fast forward many decades, to I think the combination of a lot of what you ended up doing was with these CATALISE papers that you published in 2016, and 2017. Because, I find them fascinating, I think that they, I think discussing those papers is, for me the way how I think I can understand like, how you see the the relationship between these different profiles, I guess. So I would start with, you know, with the second CATALISE paper , the 2017 one, that's where you define the term developmental language disorder or DLD. Dorothy Bishop 22:55 Yes. Stephen Wilson 22:56 And kind of defend it against other possibilities and relate it to other diagnoses or syndromes. Can you tell ,me like, how, was that the term that you came in with? Or did that emerge from the consensus process? Like, how did you get to DLD as the central defining concept there? Dorothy Bishop 23:15 Well, I have to say, when you say I did this, it was very much not me, in the sense that this was the whole idea of this was to use the Delphi method to get a consensus. And I was a moderator. And so, what, what I was intrigued by this method, and I had never, I mean, it's really going in a very different direction for me, because it's more sort of qualitative research. But I was recommended it by a colleague who works in primary, Trish Greenhalgh and professor of primary care, or general practice, or, anyway, she's a sort of doctor interested in these sorts of issues. She suggested we use this method because I was explaining to her how I was working on a condition and nobody could really quite agree what to call it. And what it was. Everybody agreed that there were kids with language problems, but there was at the time, I was getting very, very frustrated by the lack of recognition, as well. I mean, no, but most people, if you went to conferences, they knew what dyslexia was, they knew what autism was, but they had never heard of what was mostly then called specific language impairment. Stephen Wilson 24:19 Yeah. Dorothy Bishop 24:20 Now, we, so thanks to Trish and with my colleague, Maggie Snowling, we sort of thought well, it would be really good to try and get some sort of consensus on this. But to do that, you have to assemble a panel of people. And it was international English speaking countries only because we couldn't cope with foreign languages, but English, mostly speech and language therapy people, but also psychologists who are experts, some representatives of charities who represent families and so on, some medics, so it was, it was quite a mixture of about 50 people, and we did it online, which I think is essential because and we did it and anonymously. So you don't get the big famous person who can bully everybody else into agreeing with them. Everybody gets a set of statements about, you know, we're about started out about 40 statements, and then you just have to say how far you agree or disagree and give some rationale. And we very much encourage people to cite evidence, not just opinion, if possible. And…. Stephen Wilson 25:22 Can we just give our listeners a couple of examples of statements? I mean, and I don't know, if you want to, if you can just kind of, I mean, I pulled out some interesting ones. Or you can just say what, which ones you thought were important? Dorothy Bishop 25:33 I mean, one might be the, you know, the concept of language delay is useful. Because language delay is very widely used in clinical contexts. I thought it was rubbish. A lot of people thought it was rubbish, and not a helpful concept and not clearly defined. Or the idea that there could be environmental causes of language disorder, you know, that's perhaps putting it too, simply. But the statements are actually all in the papers as well. Stephen Wilson 26:01 Yeah. Dorothy Bishop 26:02 My memory is not so good. But it would be, you try and break it up to things that you want to get people to either agree on, or if they disagree to, to really get into the disagreement as to what's the reason for it. Stephen Wilson 26:13 Yeah, let me read out a couple of them that I that I kind of pulled out as ones that particularly interested me, just mostly so that our listeners get a sense of like what this is about. So for instance, statement 12 from the first paper, is: ‘there is no clear cut off that distinguishes between language impairment, regardless of its cause, from the lower end of the normal variation of language ability’. That's a fascinating concept. So much debated, right? And so you've kind of like, you've just kind of like summarized it in the statement. And then people discussed if they want to include that in the final version, when in this case, they do. Dorothy Bishop 26:45 Yes. Stephen Wilson 26:45 Another example, just to, you know, kind of give the scope here. This is another really interesting one, number 23: ‘In general, language impairment should be identified, regardless of whether there is a mismatch with nonverbal ability.’ And it goes on and fleshes it out a little bit more. But again, that's getting at a really fundamental question of like, whether we want to define a condition that has to be disproportionate to language, or whether we wanted to find a condition that involves a language impairment, rather different takes. So, that's just to give us an answer. I mean, like, of the statement, yeah. Dorothy Bishop 27:19 And those are some of the things that were highly, some things were more controversial than others. But yes, I mean, it was so that you, if you read the early literature, and which I had sort of, first of all taken on board, you know, it was like, there's this little condition out there, which is developmental, and those days developmental dysphasia or developmentally aphasia, as if it's completely separate from anything else. And you know, and it's, and these kids have got perfectly normal IQ, but we're terrible language. And I think what gradually happened with many people over the years is they thought, you know, this is unreal, this, we're not seeing kids like this, we're not seeing this clear cut disorder, it's, it's actually exactly the same with dyslexia. They're having all the same debates and discussions, you know, have you got a very clear cut condition with very clear cut boundaries? Or is it really just that there's a lot of variability and at one end, you've got these kids with disproportionate problems. And also, you know, what was happening is that lots of kids were not getting any help, because they didn't have a big enough gap between their nonverbal ability and their language skills. And so they were in a sort of no man's land, nobody was giving them any help. And I think one of the things that emerged from this whole exercise was the realization that if you're going to talk about terminology, the question is, what are the implications? And why are you doing this? And there's a sort of scientific reason, which is to do with, you know, the reality of these categories in how they map onto, you know, scientific biological reality, which I think is impossible to, you know, we're not in a state of knowledge to perhaps get there. But then what is really urgent is that these categories are being used to decide who gets help, who gets access to speech therapy. And to be honest, I mean, that is often, you know, that is the critical question that I think most of the people on our panel were much more concerned about. And at the end of the day, saying, yeah, these these labels we're using, we're not coming up with reified real, you know, categories, we're using this as a politically expedient category to say, If children have these characteristics, they would be worthy of help. And if you're planning how many Speech and Language Therapists you need, or how many special education facilities you need, you know, you should be using this sort of definition. So anyway, the term we ended up with, that was the most controversial of all the get, it was a thing where we got least agreement even right at the end, after lots of debate. I was not terribly keen on any of the options actually, but I wasn't pushing for developmental language disorder, even though that had been the term I used in my way. Back in the 1970s, in my PhD thesis, but the some of the Americans really wanted to stick with specific language impairment. Stephen Wilson 30:08 I bet they did. (Laughter) Dorothy Bishop 30:11 That’s not, I mean, one of the one of the rationales for DLD, was alignment with the DSM and ICD. So the official classifications are used. SLI doesn't feature in the Diagnostic and Statistical Manual that, you know, the medics will use and indeed it is in use for insurance, I think in the States, you have to give a DSM code. Stephen Wilson 30:38 Yeah. Dorothy Bishop 30:40 In the UK, we’re less wedded to all of that stuff. But even so, I mean, most of the I was amazed. This was a real discovery doing this project that most of the speech and language therapy people didn't use the term SLI. you know… Stephen Wilson 30:53 Yeah. I think it maybe an American academic term more than anything else. Dorothy Bishop 30:56 Yeah. Stephen Wilson 30:57 I mean, I think it's got something, it's got a problematic assumption built into its name, right? That it's specific. Dorothy Bishop 31:02 Yes. Exactly. Stephen Wilson 31:02 And I guess that's why you were probably never going to land on that one. But did you have a different term that you were pulling for then then DLD? Or? Dorothy Bishop 31:11 I can't remember what it was now. I, oh, no, God, my memory is shot. I dreamt up something that I thought was perfect, and nobody liked it. Stephen Wilson 31:27 Okay, fair enough. So, okay, it's really interesting to hear your perspective that a lot of the outcomes of this paper are not coming from you, but it is kind of emerging from this consensus process. Dorothy Bishop 31:38 Yeah, I think, I think I obviously, there’s an influence. So it's from the moderators. So Maggie, and I as moderators, this process goes through several rounds. So, you go through round one, and then you feed back to people all the results. And then they can either stick with, if they can see if they're an outlier, and if they are, they can defend their position, or they can switch. And also, we kept saying, you know, if you've got a strong evidence for your view, you know, give us the evidence. So, we could look at what evidence people were putting in as well as, you know, how strong, what the strength of feeling was. And it was interesting to see that sometimes there were very strong disagreements, that were not real disagreements, they were just people in, you know, the way we'd worded an item that people were putting in one way or another way, or coming at it with very strong prejudices one way or another. And we could actually resolve the disagreements by changing the wording. And that was great. And we could do that. Or we sometimes had two constructs bundled up in one item, and people agreed with one and not the other. So you had to, I learned that with Delphi it's really important to have just one idea that people are agreeing with at the time, and then you can put them together. But yeah, I mean, we did, I think, I think, you know, originally, I was thinking the role of the moderator would be just to integrate the whole thing. But it did require a lot of discussion between us as to you know, how to sort of modify items to see if we could get consensus. And also, I think we were bringing in, obviously, you're gonna be bringing in your own prejudices at one level. So there must have been some arguments that weighed more heavily with us than others. But we were trying really hard not to impose anything because we knew that if we want the community to, you know, go with this, we've got to bring the community with us. And we had to, you know, the people who are probably least satisfied with the end result was some of the education people there. The problem there is that in the field of education, there's general resistance to labels of any kind. I mean, there's and there's tends to be, tends to be resistance to what they regard as medicalization, some people are still very unhappy with the fact that we're talking about developmental language disorder, they think it's not disorder that's very much in line with the neurodiversity movement. People don't like that. So, I mean, it may be that it'll all change again, but we did find that we could resolve things. But yes, it was very much, not my, I mean, obviously, I've been thinking about all these issues for years, and it had, you know, and I think, you know, would have had an influence on how the whole thing was structured in the first place, and how it was all pulled together. But I think if I had come in and said, This is how you should do it. A lot of people would have just dismissed it. You know, why should we go with this psychologist who's rather biologically oriented psychologies? You know, who's not, you have to take other people's views into account. Stephen Wilson 34:36 Yeah. I wonder if did the did any of the sort of SLI, I guess you don't really know who's who, right? But I'm wondering if the SLI, were you able to bring that research community along at all? Or did you or do you feel like they maintained… Dorothy Bishop 34:49 Most of them. There was, there was, there was some problems immediately. So basically, after, you know, the exercise and the paper was published, there was quite a lot of pushback from Mabel Rice who has, is very identified with the concept of SLI, and who did write some papers sort of disagreeing, even though she’d been on the panel, but she had been sort of out voted, and I think some of the people working with her. And, you know, that's out in the open. I mean, I wrote about it on my blog, and and she wrote her papers. And I think she was she was very unhappy. And I mean, quite a lot of, some of the arguments that she put forward, that other people put forward was sort of saying, you know, we've got decades of research using this term and now you're sort of saying, we'd have to forget all of that. And we were trying to say, well, not really, because the overlap between that construct and the DLD construct is sufficiently greater, you're not saying you forget all the previous literature, we're, you know, there are some differences. But there it was. Also, there were people who were sort of trying to make out that we're just sort of saying any child with whatever level of intellectual ability is now in the category, which is not what we were saying as well, you know. Stephen Wilson 36:01 Yeah. You know, it's quite clearly distinguished, it's distinguished. Dorothy Bishop 36:05 But I think I think there were people who were, who were definitely not happy, but I'm beginning to feel, I mean, in terms of the published literature, it's, it's been very encouraging to follow the use of the terminology in the field. You know, you can just look it up on Web of Science or something and see that it really has changed since catalyzed. So it does seem, it's probably been the most impactful piece of research I've ever done in terms of changing… Stephen Wilson 36:29 It has been very impactful. And it's not just about terminology, either. I mean, I think that's an important point to make. It's about, as you said, it has like real world sort of clinical service delivery implications, but also has like, scientific implications, because, you know, if you're studying a group of people, you need to decide who that group are. Dorothy Bishop 36:51 Yeah. Exactly. Stephen Wilson 36:51 So and so yeah. Can I, if I can, can I summarize, like, how I understand your definition of it, just because I know, that's kind of what I want to ask you about, like, the causes of DLD. So, just to make sure I'm understanding it correctly, you know, it means, it, it’s distinguished from, you know, say, for instance, kids with autism, would not be primarily DLD, you'd say they have a language disorder in the context of autism. Dorothy Bishop 37:20 Yeah. Stephen Wilson 37:20 So you're really preserving that term for kids where there's not a, you know, an obvious explanatory, external factor in their language abilities. And interestingly, you're also distinguishing it from things like childhood apraxia of speech. That’s not included. And then you've got a bit of a squishiness, about phonology as far as I can, in my reading, like, so phonology is part of language in your view. But, and obviously, it is. And so it could be part of the DLD, but you also define DLD that has to, a child, for a child to be DLD, they have to kind of have a poor prognosis. And your in your position is that if somebody has an isolated phonological disorder, they don't have a poor prognosis. And so that child should not be considered to have DLD. Is that, is that how you think about it? Dorothy Bishop 38:14 It depends. I mean, this is where it does get squishy. I mean, I think that's actually a brilliant summary. I'm very impressed that you, as, as a non DLD person, you've, you've sort of absorbed it all and made it very succinct. But I think the only thing about, the thing we struggled with is that there's a lot of children with so called delayed speech or with with phonological difficulties in their preschool years. And these, these do resolve. Sometimes they're treated, sometimes they're not treated. But you know, if under the age of four, you've got a child is not this phonological system is not, you know, quite all there. We were very reluctant to have that termed a disorder, because of the implications of the term disorder, which is that, you know, this is something serious that needs, you know, sort of, it's likely to persist in the long term. But we did regard phonology as part of language. And I think at the end of the day, I feel I don't think I've ever seen a child who had obvious problems expressively with phonology, beyond the age of about four, and who hasn't didn't have anything else abnormal about their language. So I thought it was fairly safe to say, if you're a young child, your problems with speech production that are evident in the phonology because some of those very young kids through I've done a number of longitudinal studies and you know, you could get these little three or four year olds who, whose speech sound systems were all over the place. And basically, if they had good comprehension, and as far as you could tell, they're expressive grammar and so on, were all okay, they were going to do fine. All our longitudinal studies suggested that after the age of five, they'd sort themselves out and be fine. Stephen Wilson 39:58 Yeah. Dorothy Bishop 39:59 In theory could have a child who had a pure for phonological problem and nothing else who was older, and it wasn't caused, I mean, you the other thing is, you know, there wouldn't have to not be a motor component in there, it would have to be really very much a sort of the the phonological system had somehow gone awry. In theory, you could have that. But in practice, given that I had never seen one like that, I sort of felt it was fairly safe to say we're not, we're not going to muddy the waters with DLD. And we're going to say, and the other whole side of it is what about phonological awareness, which, of course, you get kids who do very poorly on tests of phonological awareness and have reading problems. And we very much said, that is not DLD. Stephen Wilson 40:39 That's dyslexia or it's a, it's related to dyslexia, Dorothy Bishop 40:43 That’s more of a sort of meta linguistic type of problem and reflecting on the nature of language. But it doesn't mean that your phonological is, you can have poor phonological awareness, but a perfectly okay phonological system that you use to communicate. Stephen Wilson 40:56 Yeah, I've kind of got this secret suspicion that like a lot of so called LD phonological disorders are really just articulatory. And the phonological changes that the kid does are actually super adaptive, right to deal with the articulatory limitations. Dorothy Bishop 41:12 When you say articulatory, do you mean mean, there's a motor component, because this is what the other was the depending where you came from, which country you were from, some people had very firm ideas about, you know, distinction between motor and linguistic aspects of speech production and so on. Stephen Wilson 41:28 Yeah, I mean, I guess I just think that like, sometimes kids haven't got certain phonemes down yet. And so they, they create these phonological rules to avoid them. And that's diagnosed as a phonological disorder. And I kind of think I just didn't have never quite understood that because I feel like it's a fairly adaptive response to not having all the speech sounds. I don't know if that makes Dorothy Bishop 41:51 There will be people that would say not having all the speech sounds is a phonological disorder. Stephen Wilson 41:54 Ah, yeah, but, but, you know, a lot of speech sounds are not developing to you like six or seven. And that's perfectly, perfectly normal Dorothy Bishop 42:01 thing. It's really more of a developmental issue. Stephen Wilson 42:03 Yeah, I think it's just a stage. I'm in. I'm like, like, you know, I'm not an expert. So that's just my secret opinion. Okay, so you mentioned just then, it's not the same thing as dyslexia. They're like, I'm sure that there's many overlaps in the population, but you can, you can definitely have one but not the other. Either way. You know, you were talking about, we're just talking about poor prognosis. So what does happen to DLD kids as they grow up? Right? I mean, they're not going around being globally aphasic? Or we’d know about it. What is it like for them as they grow older? Dorothy Bishop 42:39 I mean, they, it's interesting, actually, because I think they disappear is what we found. So again, one of the great chances I had was to do various longitudinal studies and follow kids through from the earliest was for up to about 16, and then into adulthood. And the problems become less obvious. So if you've got a four year old, who has sort of language problems, and you know, every time they open their mouth and say something, you can tell it's sort of poorly formed. And it doesn't necessarily make a lot of sense. And you can't necessarily, you know, quite often at that age, they do have problems with the speech production side of things. When they get, what I found was when they got to the age of eight, some of these kids, I would go and assess them, and they had clearly still got language problems, but they didn't stick out any longer. The teacher would often say, why are you seeing this chart, because they couldn't pick up anything particularly odd about them. What you could pick up if you tested them were two things. One was that they would have comprehension problems, sometimes with really poor vocabulary, you know, on something like the BPVS, British picture, vocabulary scale, and it's multiple choice thing. And on TROG. And then interestingly, if you've got them to do a narrative task, where they just had to retell a little story or something, you'd find that the language structure was just very simple. So they were not doing, you know, that they will often most of the utterances will be just one clause. So wouldn't get, you know, subordinate. And, of course, if you think about, you know, but you're being limited in that way, when you talk, you don't sound odd, but your ability to express complex ideas is very limited. So they would then go on to do very poorly at school. And typically, you know, then ended up if we followed them into adulthood, you know, we're in not, you know, not very well paying jobs, that so it, you know, they would be saved if they could find something that spoke to this, and this is something I very much would always advise parents is to find something your kid is really good at to just don't spend all your time beating away trying to improve language, which, you know, yes, they need help with language. But if you can find something your kid is really skilled at and enjoy doing, you know, foster that for heaven's sake, because then they have a chance of sort of finding something a) to get some self respect and to sort of, you know, enjoy life, but also might end up as a career path. So you might end up being really good at, you know, being a cook or doing art or being, you know, some sort of practical 3d manipulation of things with the client, I can't do you know, so there were, there were some kids who you could see could foster a particular skill that they had, and that would give them a path in life. And typically, these kids were not too bad socially. I mean, they were, you know, able to, I mean, they might be a little immature for their age. But it was in that regard, rather different from the ones who we, at some point did some comparison of following up kids with autism or with the more obvious pragmatic problems. And those kids actually would sometimes do well, and go to university and everything, but then ended up socially quite isolated. Stephen Wilson 45:51 Right. Dorothy Bishop 45:51 So there is, again, this sort of, it's not really a double dissociation, but you could see instances where the, you know, the formal language skills were really the main problem, and but the socialization side of it was fine. Stephen Wilson 46:05 Uh huh. So the DLD kids, you know, it's kind of really affecting them in ways that are sort of obvious. I mean, it simply means language. So they're, they're a little bit socially behind. And it's going to affect the education, but they're going to sort of not so much grow out of it, but grow into being low end language users. And that's that getting back to that individual variation that you said, that you have been fascinated with from the get go. I mean, I guess I can sort of see the point of view of the people who say, why are we calling this a disorder? Right? I mean, if this is just the, this is just the 10% of people, or five to 10% of people who are going to be the lower end languages, are they really disordered? Dorothy Bishop 46:42 Well I think that the only point is that the term is the pragmatics of it in the sense that if you, some of the people working in education would would just want to say, you know, these are kids who have speech, language and communication needs. Problem is that will not get you help, you know, if you talk to politicians and say, well, there's kids who've got needs, and they say, yeah, you know. I have to say, to me, you know, the disorder term is very much just a practical way of flagging up, this is a problem that is serious enough that this kid is likely to need help. If they don't get help, they're just going to sort of troll along, not just at the bottom of, they're probably going to get further and further behind other kids. If you give them help, you're not going to it's not a magic cure. This was the other thing that came out from a lot of the discussions around the catalise thing was some of the practitioners I think, had a huge sense of relief, talking to researchers who are sort of agreeing with them that you're not going to turn these kids around with an eight week therapy session, which is, what was implicit belief in, in the British system was that, you know, that's what you diagnosed, and then the child was killed, and then they were out. And it meant that, you know, they felt they were failing with these older kids, and you're sort of having to say to them, no, you know, you can do quite a lot to make them adapt better. I mean, I guess this is probably not that different, in some ways, from working with people with an acquired aphasia, that's, you know, really not going to shift. The question is, you know, you can teach people to adapt to the situation, and make the most of the skills they've got. And you can, you can do useful things. But if you could go in expecting that, you know, after two weeks or two months of seeing you, they're going to no longer have problems, then you're going to come away very demoralized. Stephen Wilson 48:26 Yeah. I mean, that's definitely the, a common point of view in speech pathology is that these things can just be like fixed if you just go in and do the exercises and that's not something that I think you'd believe, if you've been researching it, and like, I do agree with you. It's, it's quite similar to the Aphasia situation. Yeah, we could, I mean, we could talk about intervention as a whole nother conversation, but I'd love to get to neural basis questions. So, you know, I know that you have published several papers over the years about the neural difference between kids with DLD and and you know, matched controls, especially investigating the hypothesis that it might be related to atypical language lateralization. But then you have a very fascinating paper from 2018, which is titled and like, sorry, I just have to like refer to the title because I love it. It's called: ‘Resounding failure to replicate links between developmental language disorder and cerebral lateralisation’, ‘resounding failure’. Not many people title their paper 'resounding failure'. But in the in this paper, and I'd love it if you could kind of talk us through it a bit, but like you essentially, do not replicate those prior findings that DLD might be associated with failure to lateralized to the left hemisphere. Can you tell us about that. Dorothy Bishop 49:53 Yes. Yes, yes. I mean, it was disappointing, but it was I sort of, I think I felt it needed that title because basically, we should look, I mean, there's been a load of research looking at handedness, which of course, is the most indirect proxy of cerebral lateralization, for language. And that literature is a complete mess, handed this associated with all sorts of associations with tons and tons of phenotypes, I think the entire literature is a mess, because it is, I think there's publication bias, and I think there's what we call P-hacking, which is, you know, people analyzing the data every which way until something comes out. So, I think, don't believe most of that literature. But I did have a sort of interest in the idea of a link with cerebral lateralization, because we'd done a couple of studies using functional transcranial Doppler (fTCD), which I think is a reasonably good way of looking at language lateralization in the brain getting away just from looking at handedness, what you're doing is you're putting ultrasound probes above the left and right ear. And they, if you angle them, I mean, this is a standard process used in Europe, by neurologists, working with stroke patients, and so on to look at the integrity of the blood vessels. So it's well established that you can use, using Doppler, you can track the velocity of blood in the brain arteries, and particularly can get the middle cerebral artery, you angle it the right way. And it's a wonderful technique, because you can sort of see the blood going swish, swish, swish through the cerebral artery on the screen, which, you know, gives us a sense of this is something real, we're measuring here. If you then give people a task, such as word generation, and one of the beauties of this method is that, you know, is much less sensitive to movement artifact than say fMRI. If you give them a task, such as generating words, beginning with a particular letter, while you're doing this measurement, you in most people start to see an increase of blood flow to the left hemisphere. And it's, it's, you know, not huge, but it's sort of 1%, something like that. It's big enough. And what I spend a lot of time doing before we went on to kids was just establishing this is reasonably reliable. And this, this approach was developed in Germany, by Michael Deppe and colleagues in Münster. But it, we were sort of really, I was really keen to take it forward, because it was usable with kids, it's portable, and it seems reasonably reliable, fairly quick to do. Stephen Wilson 52:36 Let’s just kind of sit on that for a moment. Like, you know, I just kind of want to like, understand what, like, what's the motivation for choosing this over fMRI, right? So fMRI obviously, well, you know, very well validated for determining language lateralization. Dorothy Bishop 52:53 Oh, I don’t know about that. I don’t know about that. I have questions about how you do it. But yeah… Stephen Wilson 52:58 Yeah, we could, I'm sure we could, sounds like we could probably have an argument about that. That's, that's all well and good. Okay, let's say like fMRI widely used for this has very good concordance with water outperforms water. In fact, when they are in conflict, and I get it, why are you using Doppler. It's for, it's because it's appropriate in kids, right? I mean, like you, it's a lot, it'd be very expensive to put this many kids in the scanner, you have to get them to a location, you have to get them into that unfamiliar environment. Those are the kinds of motivations that lead you to go to Doppler, or… Dorothy Bishop 53:38 It will cost hundreds of pounds for each child, quite apart the practical difficulties. So, once you've got a Doppler machine, you can run around and all you have to pay for is the gel. I mean, it's I mean, the costs are vastly different than that allows you to scale up and to get this is why we could do the resounding paper because we could get sufficiently large numbers that we could, you know, get, get power, to be confident that if we got an null result, it was an null result. And that's why I use the term resounding. So, there is no way I could have studied that number of kids. And just just to get the kids into the situation and through, I mean, even with Doppler, we lose data from kids that wriggle around and what don't like the headset on and so on, but it's just a much more child friendly method, but even with I mean, we've done a lot now as well, just with adults and looking at links with things like how did is it just allows you to do things at scale and to get literally hundreds of people tested. Stephen Wilson 54:35 Yeah. it's yeah, it's fascinating. And you know, I guess it's like, convenient, right, that the MCA, the middle cerebral artery is the biggest artery and, and that that happens to feed both the frontal and temporal language areas. So it's kind of like conveniently organized that, that you know… Dorothy Bishop 54:53 The most recent study, yeah, the last study I did before retiring, we wanted to compare Doppler and fMRI laterality, which is why I'm making noises when you say, oh, it's well established, because I think, and unfortunately, we were sort of completely bought by the pandemic. But we have got data on about 50 people where we've done both, and I'm in the moment in the throes of doing comparative analyses, but the agreement is good. Stephen Wilson 55:20 Yeah. Dorothy Bishop 55:22 I'm not happy, though, with the method that's usually used to get out of laterality index, and I tell you about why that's supposed to… Stephen Wilson 55:29 Yeah. I saw your I saw your recent paper, still in press, I think, in neurobiology of language. It's pretty, I think it's pretty awesome that you've retired, and you're still having first author papers in Neurobiology of Language. You know, and, you know, it's interesting, like, you know, this makes me think, Oh, I wonder if we should be doing Doppler with our stroke patients, because, you know, we only scan about, you know, of the people who we follow. longitudinally in my lab, we probably only stand about 40%. Because of practicalities. If we had, if we had Doppler in our arsenal, we could be sampling a lot more people, at least to look at the lateralization question, Dorothy Bishop 56:09 I don’t know how much it would be, I mean, you’d obviously have to talk to somebody who knew about these things. But given that what you're measuring is blood flow, flow through the middle cerebral artery. I mean, I don't know what happens if you've got, you know, compromised blood vessels and how that would affect you. Stephen Wilson 56:25 I mean, that affects fMRI too, obviously. Right? Like, I mean, they're both hemodynamic signals. So yeah, that's a question that's You can't get away from that too easily. Dorothy Bishop 56:34 It might be worth though talking to your local people that do it. Because I mean, it is, as I said, widely used diagnostically, but usually just unilaterally, so people would normally just be looking at it to it absolutely explore, you know, what the blood flow is. And you can look beyond the middle cerebral artery, you can you, can actually image depending on how you angle your probe you can you can look at other arteries. And I know, you know, the people who use it clinically use it very differently from the way we are using it for research purposes. And there's, a there's quite a lot of good knowledge out there. So, it might be worth integrating with those people. Because it would be indeed, you know, intriguing if you could get some value out of it. Stephen Wilson 57:17 Yeah, no, I think I do, I think it's something worth looking into. But yeah, so getting back to your paper, though, like so you've got, you know, like a large sample size here, right? You got 107 kids with the DLD, and 156 matched controls. It came out of a twin study, but that's not really relevant here. Right? And then you kind of modeling that. But obviously, you've got some other stuff going on there that we probably didn't have time to talk about today. And, you know, you do your Doppler lateralization. And you are expecting I'm sure to see more right dominance or bilateral ality in the DLD kids compared to the controls. Dorothy Bishop 58:01 Yeah. Stephen Wilson 58:02 And that wasn't what you saw. (Laughter) Dorothy Bishop 58:05 No. I can't remember. I mean, basically, completely, it was like about as null as it could get and but we did find the left lateralization. I mean, if we'd found, you know, no left lateralization, then you'd say, well, it's just a rubbish methods and it doesn't work with kids or something. But, you know, we reliably see this. And you can also you can get out measures of error of measurement in individuals, which is nice. It's another thing I'm at the moment absolutely fascinated by sort of, how far can you categorize an individual person using this method as either lateralized? But in general, I mean, yes. It's, I it convinced me that, that there's absolutely, I would need a lot, a lot of evidence now to shift me back into the position of thinking that, you know, atypical lateralization had anything to do with DLD. Stephen Wilson 58:56 And how did that make you feel like, given that you had prior findings in that direction? You, I guess you mentioned in your paper that we're talking about that you think they're false positives? That was that like, shocking, or were you just kind of like you just rolled with it and just kind of adapted to new worldview? Like, what was that like? Dorothy Bishop 59:18 It was, it was, it wasn't entirely surprising, because I do feel an awful lot of the literature is sort of flawed in that we are, I mean, I’ve tried my best not to, you know, be biased in how I was doing things. But I think we're all hoping to find nice, interesting, exciting findings and probably more willing to, you know, take seriously evidence that goes in one direction than another. So, I wasn't, you know, I wasn't dismayed. And the other thing was, it had always been a puzzle to me. It resolved one big puzzle and the big puzzle was that there's study after study, in typical people, that is tries to compare various cognitive functions in those with left or right lateralization. It typically finds nothing. I mean, this is this, to me is absolutely intriguing that, this is a sort of population bias towards left hemisphere language, you'd think therefore, that left hemisphere language might, in some sense, be good, a good thing for how it works.. But in fact, if you look at people who have atypical lateralization, who was not selected as having a disorder, but just taken from the general population, it's very, very hard to find any evidence that it's a disadvantage in either language processing or anything else. As far as I can see, that's my reading of the literature. Yeah. So in a sense, resolved that and said, because I'd been trying to explain, in fact, in one of my papers, I went in great details about how you might have an association and disorder group, but not see it in a big population. And so in a sense, it just sort of it simplifies life if it's a false positive. And I think it's really important, though, not to be so wedded to a theory that you're not willing to change if the evidence goes against. I think, you know, that's perhaps another thing I was trying to think, you know, let's be a role model for somebody coming out and saying, I thought this, evidence goes against it, let's move on. Because otherwise, I mean, I think, you know, we get really, if people get so stuck and wedded to their theories that they will not move on in that way. It's very bad for science. Stephen Wilson 1:01:16 Yeah. And that, I mean, that's kind of almost takes us to our next theme, right? I because I told you, I wanted to talk about your interest in meta-science. But I can't help but as a, you know, as a huge lateralization nerd, and, and imager, I have, I want to mention that I do think that I do agree with you that the Doppler findings there, you know, you have that validity of demonstrating the left lateralization certainty. And I, and I think that it makes your point, absolutely. But I would, I do think that you show a more bilateral language network with Doppler than we do with fMRI. And I don't know to what extent it's kids, but I don't think it's because it's kids, because I compared your findings to Marison Berl and Bill Gaillard’s work, which I think do the best, I think they do the best imaging with kids with fMRI. And they show like a lot more left lateralization than what you see with Doppler. So I'm wondering if not, you know, I think the directionality of the, I don't think it changes any of your conclusions at all. But I wonder if they animate, so I guess I'm just asking you about your data that you mentioned that you're working on, where you actually got both measures in the in, in the kids like, do you see that? Or, no, like, do you see a more bilateral picture coming from Doppler? Dorothy Bishop 1:02:39 Well, gosh! So you're, what you're referring to is the proportion of people who are left lateralized, I guess. Stephen Wilson 1:02:48 Yeah, I’d say like, if you look at Berl and Gaillard’s work, most kids of the age that you're studying, which is like six to twelve are left lateralized. Like really like, you know, 90% plus. And that's not what you find you find more like, you know, 70%, and then there's some some bilateral and some right. And so one possibility, and this is what, you know, I know that Berl and Gaillard and Elissa Newport and all those guys, they will think that they kind of language becomes more, you know, more lateralized as we get older. So one possibility is it's just a population issue, but I don't really think it is. I think it's a methodological difference. Dorothy Bishop 1:03:25 Yeah. I mean, so this is the other big issue that I'm absolutely obsessed with is measurement error. I mean, is it measurement error? The, so, is it just that, it's noisy, right? So if it's noisy, you'll sometimes look at if you're one side, and now that so at the moment, I'm doing a whole load of analyses, where I'm looking at an individual level with, for start, we developed a method of analyzing Doppler data that gives you a smaller standard arrow, because we're using general additive modeling, rather than just taking the difference in blood flow in the two sides. And that allows you to absorb more error. I haven't tried that kids data, and it will be quite interesting to see if it made a difference. But what is fascinating to me is that we've now got fMRI data, where we can look at different brain regions, and compare different brain regions on a whole range of tasks, actually, not just language generation tasks, and see how they compare with the Doppler and the, this is adults. But in these adults, there's pretty good agreement between what you see with Doppler and what you see, in terms of particularly frontal activations. What is interesting is there are some tasks where particularly more receptive tasks, which look more bilateral on fMRI, when you look at the individuals, it's not that everybody's using both sides of the brain. It's rather that you've got a mixture of people who are left lateralized and right that arise. And I'm still trying to get my head around that. But you’re, we're actually able to compare, you can really sort of if you can get a handle on how much somebody's score is just error related and how much it's, it's more a genuine bias to one side, whole load of new questions then sort of starts popping up about whether the group data that you get out of fMRI is really a reflection of what's going on in individuals. And in general, you know, particularly for things that are thought to be bilateral, I'm getting the impression that in general, that's not so. But yeah, I mean, I guess I guess I'll have to look back again, that you make me think I'll look back again, at that the Berl and Gaillard work, but… Stephen Wilson 1:05:45 Yeah, I just, I just kind of looked at them, because I was like, Oh, your your data looks less lateralized than I would have expected. Like, like, again, like not that not that reflects the point. Because you should have seen, I mean, you should have seen life. And if you had like, even if you had a measure that wasn't revealing, as much lateralization as there may be, and who's to say which one's right, right? There's no gold standard here, necessarily. But yeah… Dorothy Bishop 1:06:11 When I was being a little bit, when I was being a little bit funny about fMRI and laterality, I mean, I am concerned that it makes a huge difference, what you see, depending on how you define your laterality index, and where you put that threshold, and you can get, you know, everything from a laterality index of 1 to 0.5, depending on what you know, if you're going to sort of say how many voxels are going to have to be suprathreshold and you change your threshold, you get a different result. So I'd be interested to go back and look at those people and see exactly how they did it. And because it's really possible to, you know, get, I think, I think it's hard to say this is the population value, if you are not sure how it would, how you define laterality in that context. Stephen Wilson 1:06:55 And of course, as you know, because you've published on this with Abbie Bradshaw, it also made us a huge amount what task he is and the task contrast. I think that we certainly agree that like, the resolution to these kinds of questions is measurement like really comes from the centre, right? We really need to work on our, on our measurement and things like validity and reliability, and really understanding those, which the field has been a little bit lacks on. Dorothy Bishop 1:07:23 Yeah. Stephen Wilson 1:07:24 But that's, I think that's, you know, that's where the answers are going to be found. And it's, I'm, I'm interested to hear that you're doing this work, and you don't sound very retired though. It doesn't sound like very retired. Dorothy Bishop 1:07:37 Well, the thing about retirement is, it's nice is that you can do the bits you like doing. And so I mean, to me, this is as much fun as doing crosswords, it's nice to have, we have data leftover partly because of the pandemic. But partly, I think, because everybody has data that, you know, they haven't had time to analyze. But I am, I should reassure anybody out there that I am also doing things like taking drawing classes, going into the cinema in the middle of the afternoon, going for long walks, you know, I'm, I'm not just sort of beavering away and doing nothing but work, but I do as much as I like. And I very much enjoy not having, you know, all the bureaucracy of running a grant and that sort of thing. Stephen Wilson 1:08:18 Lovely. Well, we won't talk about your walks in the park, on the Language Neuroscience podcast. (Laughter) So, you know, you've really shifted a lot of your efforts, like, you know, like we've just been talking, obviously, you're still doing some really detail oriented work on these questions that have you spent your career on. But you've also shifted a lot of your efforts into meta-science and you have this cool paper from 2019 called: The Four Horsemen of irreproducibility, or something like that. You describe those as being publication bias, small sample sizes, P-hacking, and HARKing. Dorothy Bishop 1:08:59 Yeah. Stephen Wilson 1:08:59 Do you still see these as being the, the big factors that are making science less reproducible than it should be? Dorothy Bishop 1:09:10 Yes and I should explain for your listeners, HARKing is hypothesizing after the results are known. The term was invented in the 1990s by a very cool social psychologist. But it's, I think those are all still problems. Unfortunately, I think the real problem and this might not affect your area as much and perhaps not mine, but the real big problem now is fraud. I mean, it’s just gone, it’s gone mad in the past year or two, with these industrial scale, so called paper mills, which advertise on the internet that you can buy, buy authorship on a paper and then they see these papers into the literature and the ones that people like me have easily picked up and which have led to mass retractions, are ones that are just crazy and probably AI generated. And you know, you can tell just from reading them that there's they make no sense. But what’s really scary is that the more people are doing meta analyses, the more they're realizing that substantial proportions of papers in some areas are fraudulent. And are not crazy, you know, I mean, they look very plausible. But typically, these are written from templates so that somebody gets a perfectly decent paper that's accepted. And then they just tweak it a bit, change the phenotype change the measure, or, in fact, often they don't change a great deal. And some of it you can see is plagiarized. But these things are getting into the literature at an alarming rate. And I, you know, I'm sort of feeling that I'm sort of standing on the edge of the cliff waving a flag saying to people, for God's sake, we've got to do something about this, because it's, it's not being the publishers are beginning to wake up, the publishers have got a problem, because they've got some of them are better than others at this, and they have a conflict of interest, they want to publish lots of stuff. So they don't really, some of them don't seem to really care whether it's true or not. But institutions are very bad at risk, you know, taking action if somebody is found to be publishing fraudulent stuff, but it's, it's getting way out of control. And just this week, I've heard from two people trying to do meta analyses, in fact, one of them has published an account of this, where you just go in to sort of look at, I think that one of them was like, you know, treatments for depression or something. And they went and did the standard thing of let's try and find all the literature, they've had 20% of the papers were fraudulent. And those are the ones that they can, you know, tell a fraudulent because they've got various signatures of fraud in them. And then there's another guy I was just emailing with who's, who's sort of, again, just sort of shocked to discover in his field, that he was doing a meta analysis, and it was a similar sort of 20, 25% of papers. So something very bad has started happening in terms of having a publication being a pathway to a career and, you know, get perhaps, in some cases, getting out of your not very nice country. But people are in I should say, it's not just people from I mean, it's been talked about a lot as authors from China and Iran, but we're seeing fraud coming from all over, you know, and it's, it's, it's in Europe, it's in the US. So, in a sense, the other problems sort of pale in significance against that, because, you know, with with the publication bias and p-hacking, you feel you've got people who are doing their best, but perhaps don't, you know, fully understand why it's a good idea to guard against this sort of bias, because that bias means that what gets into the literature is not a realistic representation of what's really the case because it's biased. But if what you've got is fraud, then what's getting into the literature is just rubbish and just plain wrong. And it's in areas like cancer biology, you know, yeah, it's serious. Stephen Wilson 1:13:02 I think, in a way, like, my field has probably not gotten the fraud yet, because it is kind of a little bit of a backwater. You know, like, if there's just, I mean, it's gonna come first for things where, like, There's big money going around. So you know, in that, in that 2019 paper, you expressed optimism, you, you, you wrote, and I quote: ‘ I anticipate that these forces will soon gain the upper hand,’ You meant the forces against, you know, the new things that will go against, you know, pre your pre registration, open science, etc, and the Four Horsemen might finally be slain. So you were quite optimistic then. Are you less optimistic now? Dorothy Bishop 1:13:39 Gosh, I hadn't realized that that optimistic? Yeah, I think I was optimistic. I mean, I think I am optimistic that measures are very much being taken against the four horsemen. And I think things like open, I mean, having open sciences is really good. I think it will help against fraud. And what I'm very much coming down in favor of is open peer review. It doesn't mean that everybody has to be named, you could have your peer review made public without you being identified if you wanted. But to me, that's one of the easiest ways of spotting fraud is that quite often it's accompanied by fraudulent peer review, because somebody from editor somewhere in the system is corrupted, and they're just putting out this boiler plate stuff, which you can just read it and you know, that's not written by a real person that's read that paper. So things like some of the open science methods, I think, will help stem, I think it might be the only way to stem the tide of fraud. A plus perhaps more serious consequences for people who commit it. But it is, it is, yeah, I'm feeling pretty pessimistic about finding I mean, in the past, if I read a paper and I thought, Oh, I don't think this is right. There's something you know, and I thought it might be from a paper mill and it might be a bit dodgy. I would look at other papers in that area and see, you know, is it in the right ballpark does it sort of more or less agree? And recently, I've been finding, if I try and do that, then I can't I don't know whether to trust the other papers in the area, you know, there's so much out there. And in some fields of study, it does seem to be that a whole genre has sprung up of fraudulent papers that are almost swamping the real stuff. So that if you were to take a vote as to, which is, you know, the most likely explanation, you'd be misled. So it's, it's scary stuff. And it's, I suppose it's in some ways, it's in parallel with what's happening in politics. And, you know, with everybody worried about fake news and not know what to believe anymore. I think it is infecting science. And I think, yeah, it's a good, good time to be in a backwater. Stephen Wilson 1:15:44 Well, yeah, I mean, but that's a short term solution, right? I mean, like, you hope, if you're working in a backwater, it's because you think that it's more important than it's being given credit for, you wish that your field becomes important enough to be a target of fraud. But um, you know, I think that, like, it's gonna be a drastic change and I'm, and, you know, like, I'm not sure about you, but like, for me, at least, like my default position on any paper, is that it's probably not right. And I think that like, with, like, I'm not, you know, I don't go into you know, just because something's peer reviewed. I mean, that doesn't mean anything, really. And it's the things you said before, like, you know, the the p-hacking, the small samples, the Harking, just the silly questions. And then, of course, right on top of all that, but like, I feel like, it wasn't like we were coming from a position where we had a great reason to be confident in papers. So, so, I almost think like, I've done a meta analysis, and it was definitely there weren't any fraudulent papers in it, because I, I went very deeply into them and I know, the know, the authors and whatnot, in most cases, but it definitely. You know, I don't think that meta analyses are the path to truth, like, because I do, because I'd much rather to read one really great study with a large sample and really good design than a meta analysis of the same number of subjects from from 70 Labs honestly. Do you? Dorothy Bishop 1:17:17 Yeah. it might depend on the research question. I mean, I think where meta analyses have come into their own is, of course, in clinical trials and things where you've got a whole load of studies all purport to be investigating the same drug for the same condition. And so yeah, it makes sense to try and integrate the literature. And I agree, even there, you know, one big well done study is better than lots of little ones. But I think, you know, there is scope for accumulating evidence, particularly in fields where it might be hard to get very big samples that, you know, maybe then if you can take, you know, 50 studies, each with 20 people, you're, you're doing better than just if you've just got one of those studies. So I think I think there's, I think meta analysis, I agree with you. And a lot of you've got to be if you're going to do one, you have to be very careful about the quality of evidence you're looking at. But I think what's happened is that people doing them have what it has thrown up problems that people didn't realize we're out there, because it's in the context of reading a whole load of papers on this topic, you can then sort of see which ones are really sort of suddenly poking out and looking completely out of line in some way. something seriously wrong. I mean, they you get things with effect sizes that are four times the effect size of everything else, well, then you start to be suspicious. Stephen Wilson 1:18:32 Yeah. And I guess we just I mean, I think we just need to, to move to a different model where we don't kind of give credence to published papers. I mean, it's it's a, it might sound dramatic, but in a way, it's almost like a return to 100 years ago, where, you know, scientists would have known all the other scientists that were working on their question, and you kind of knew who was, who was doing trustworthy work. Dorothy Bishop 1:18:57 Well, I think if you're in an area, again, if you're in an area where that is the case, that is good. Of course, the problem, then is that you do get people who feel that that's then a clique, and how can they break in and they might be trying to do perfectly good work in some place in the back of beyond and find that, you know, nobody will take them seriously because their work isn't known. And in fact, you know, they're just fine. So I think we've got to be quite careful to not, you know, you don't want to be exclusionary, and I'm quite worried that, you know, it's getting to the point now with fraud that a lot of people I know, just saying, well, if a paper is from Iran, it's likely to be dodgy. If it's from China, it's likely to be dodgy. Well, this is not very good if you're somebody who's trying to do honest science in those countries. So, I think we have to be very careful to encourage those who are trying to anti science but I mean, to be honest, I do feel in some places it does seem as if the people who are doing the fraud are right at the top of the tree, and you know, if you're honest, you're not going to survive in that system. So, I think you know, some, there's going to be some real, I don't know, some heads have got to fall at quite high levels before some places are going to be able to rehabilitate their reputations. We're, we're in we're living in very strange times. But I think somebody was saying on on social media that, you know, wouldn't it be good to go back to the days when you, there were no journals and you know, you could just put out your, when plank wrote his masterpiece, you know, he just sort of put it out there and didn't have to worry about peer review or journals. And he still think, well, that was great. But in Planck’s time, there were, there weren't all these fraudsters. It's just getting very hard to work out what’s signal and what's noise. And I agree with you that, you know, I know who whose work I can trust, I think, but that you don't want to have a system that's close to newcomers. Stephen Wilson 1:20:52 Right. And that's, yeah, and I don't I don't think that's insurmountable. You know, I think like, you know, every, every so often, I wish, not, not even infrequently, I'll encounter a paper from somebody that I've never heard of before and it'll just like, the quality just shines through and you just realize, oh, here's the new person that I, I want to listen to, you know, and I don't, and it doesn't matter what country they come from, it can be anywhere. Like, you'll just know those papers when you see them. Dorothy Bishop 1:21:25 Or a conference. I mean, I know you've on your podcast, you featured some quite junior people. And you know, I think you've picked up people who are clearly in your mind, you know, going to be interesting and doing good stuff and that’s great that you include those people. Stephen Wilson 1:21:38 Yeah, I mean, like, I think that the people are doing great work at all career stages. And I'm obviously I mean, the future 'you' has to be starting out somewhere now. Right? So I, if I can figure out who that is, I want to talk to them. Well, thank you very much for taking the time to talk with me. I really appreciate it. Dorothy Bishop 1:22:06 Thank you for inviting me. It's, and I'm very impressed at how much homework you've done. And you, you ask all the right questions. It’s fantastic. Stephen Wilson 1:22:14 I mean, I've been following your work for a long time and you know, because I'm really like, interested in laterality and I, you know, I first encountered it through your, your colleague, Abbie Bradshaw’s papers that were very much like relevant to what I was interested in and still am interested in. So that led me to, you know, your wider body of work. So yeah, I've been looking forward to talking to you about all this stuff, and I only wish we had more time. But I hope we can catch up sometime in the future. Dorothy Bishop 1:22:42 Right. Okay. Well, lovely talking to you. And thank you very much. Stephen Wilson 1:22:46 You too. Take care. Okay, well, that's it for episode 29. Thank you, Dorothy for coming on the show. As always, I've linked the papers we discussed in the podcast notes, and on the podcast website at langneurosci.org/podcast. Thank you to the journal Neurobiology of Language for supporting some of the costs of transcription. And thank you, as always to Marcia Petyt for transcribing this episode. Bye for now. See you next time.