The coming years are likely to see substantial changes in the nature of work and in how people find jobs. Over the last two decades, much of the job search and recruitment activity has moved online.
In this Keynote, Professors Michèle Belot and Jason Sockin will discuss ongoing research on how digital technologies can be used to improve the process of matching jobs to people. Professor Belot, who leads the Cornell Job Search Lab, develops and evaluates digital tools aimed at forming better matches and navigating structural changes, while Professor Sockin brings expertise in labor economics and the shifting dynamics of work.
This session will explore how technology can drive better job matches and address the challenges of a transforming labor market.
What You'll Learn:
The Cornell ILR Job Search Lab
K062325 - The Future of Work Technology
Chris Wofford: [00:00:00] On today's episode of Cornell Keynotes, we are exploring how digital technologies can revolutionize the way we match jobs to people. Now, job search and recruitment are nearly entirely online. So the challenge now lies in using digital tools to create even better matches between people and jobs, and it's the combined research and tech that is making this possible.
Professors Michèle Belot and Jason Sockin from the Cornell ILR school join us to discuss their groundbreaking research on improving these job matching processes. Professor Belot, who leads the Cornell Job Search Lab, develops and evaluates digital tools that are aimed at forming better matches while Professor Sockin brings his expertise in labor economics and workplace dynamics.
So these are two colleagues who are very well matched on this podcast. The two explore critical questions about the future of recruitment. Again, how can we use these digital tools to create better job matches? But beyond that, what are the [00:01:00] pitfalls and benefits of using AI and recommendation systems?
And finally, and most importantly, how can organizations evaluate and start using this new tech? So our guests break down the real opportunities and challenges of connecting talent with jobs that are the right fit. So there's something here for both employers and would-be employees. But before we get into it, I want you to note that you can find information about the Cornell Job Search Lab, and ways that you can participate in Michèle's ongoing research down in the episode notes.
So now here's Michèle and Jason.
Jason Sockin: What, if I might ask, really motivated you, would you say, to study labor economics and in particular job search?
Michèle Belot: Yeah, so maybe I'll start actually with the fact that I did grow up in a country that had a very high unemployment rate.
So I come originally from Belgium, and I'm coming from the southern part, which is the part where they speak French, as you can probably hear from my [00:02:00] accent. And, uh, the part where I grew up was really in transition, so they used to have a really thriving steel and mining industry. And, um, it basically tanked, uh, you know, a few decades ago.
And so when I was growing up, the unemployment rate was around 30% and very high share of long-term unemployed. And so you cannot be insensitive to what is happening around you. You really feel that we are not very good—and this is not only true for my region, but I think there are many examples across the world—when there is an economic shock or when there is an industry that stops thriving, [00:03:00] we are not very good at transitioning to something else. And so I've been really interested in trying to understand, well, how can we actually design, you know, institutions? What kind of policies can we put in place to help people transitioning and to help also the regions transitioning to something else?
Jason Sockin: Michèle, I actually didn't know that background and it's so powerful. I wanted to ask you actually, just to follow up on that, 'cause I think if you look historically in the US we've never seen 30% unemployment, at least since the Great Depression. I think Great Recession or COVID recession, we get up above 20% depending upon reporting issues.
But can you maybe describe for people what it's like being in a society where there's 30% unemployment? Like what is that like growing up in?
Michèle Belot: So I think what is really difficult is this lack of prospects. So maybe in a way you don't feel so bad because there are lots of other people who are in the same situation. So that's maybe a little bit, [00:04:00] you know, a boost. But on the other hand, it feels very grim. Right? And it feels very grim even for those who are employed and who actually have jobs because you wonder, well, how long are you going to keep your job? And so I think there is this very grim atmosphere and this lack of, um, you know, sense of how can we actually solve this? And I think that, you know, of course one can get a little bit frustrated by the policies that are implemented. I certainly feel that one could do more to actually try to help people get back on track and to really transition to other things that are thriving.
But so yeah, so that's, uh, that's basically my experience.
Jason Sockin: And I know you've done a lot of research sort of in that realm, right? Trying to think about prospects and how to help people find jobs. And I'm sure we're gonna talk at length about that throughout this keynote. I was wondering maybe to give the audience a bit more background, can you tell us maybe a bit more about one of your most recent publications? [00:05:00] What was the paper about and what did you find?
Michèle Belot: Yeah. So I'll talk about one of my most recent publications that really relates to that topic with—which is joint with Philip Kircher, who is also a professor at Cornell and with my long collaborator, Paul Muller, who is in the Netherlands.
And we did actually a study in the UK, um, in the Midlands region—sort of the middle of the country. And we collaborated with a private company that is really charged by the UK government to place job seekers who have been searching for a job for a long time. So usually, you know, most of the time it'll be longer than a year.
And many of them will have other challenges. So it's not only that they are searching for a job, but they might also have other challenges like a disability and so on. So they really try to help them on various fronts. But so they are tasked with trying to help them indeed to find a [00:06:00] job. And so we worked with them because they designed a job search platform that was meant to help them to navigate job search and to identify jobs that might be suitable for them. And so we actually thought about, well, how can we actually indeed use these tools to make recommendations that might be helpful? And so we were working with a population that sometimes never had a job, you know, they—it might be the case that they never worked before.
We didn't necessarily have an anchor of like, you know, job experience or things like that. So we also included what we call a wheel where they could actually choose between 18 different occupations that did not require a lot of qualifications, but that were high in demand. And so that might be things, you know, in the food preparation industry or packaging industry or things where you need some training once you take the job, but it's not that people are necessarily selected based on their prequalifications. And so we had that. And then we also had, for those who had some work experience or actually some [00:07:00] qualifications, we also potentially had another anchor—an occupation they might be interested in to start with. And we used that as an anchor to make suggestions of other occupations they could consider that basically relate to those. The way we identify those is by looking at past transitions that other people have made, and we know that people who have had a particular occupation might then have, you know, a different occupation in the future. And we can use that information to make recommendations that are hopefully sensible. And so what we did is we did a randomized control trial.
So that means that we didn't give access to the tool to everyone at the same time, which is, you know, something that makes sense because you first want to understand whether your tool is gonna be helpful or not. And so we used that to compare really people who had [00:08:00] access to the tool to those who didn't. And this was actually randomized, so that means that whether you actually got access or not was determined at random. It seems to us it's also the fair thing to do, right? You don't want to choose between people using, you know, sort of ad hoc criteria, so you get it or not. And it's a bit random and at the end of course, you know, the idea is that everybody would get access to it if it's helpful. And so what we saw is that there was actually a really significant and quite substantial impact on their probability of having a stable job. So that means a job that lasts for at least six months. And so the effect was quite large.
There was an increase of 30% in the probability of having a stable job. And so that I think was quite encouraging. And actually the company we worked with decided to roll it out and implement it on a larger scale afterwards.
Jason Sockin: No, that's fantastic. [00:09:00] I mean, it's very apropos for what we're discussing today, but I think also just emphasizing that, like, I think one thing that stands out to me about that is it's not just finding a job, it's finding a stable job. Right. And for a job to be stable, both the employer and the employee have to be happy with it, right?
And so I think that's a real improvement of like, something in the match process is improving because of that platform that you're providing. Can I just ask you a quick follow-up question, Michèle, about the paper? 'Cause I think sometimes the word occupation, we use it so much, but that can be a little bit hard for people to recognize.
Like, what is an occupation you're choosing amongst these? Can you describe what an occupation is and then maybe just to give people a sense of what kind of transitions people are really making when they're looking outside their occupation? Do you have any, like, off the top of your head, a few different types of transitions that people really made when they got these more stable jobs?
Michèle Belot: Yes. So actually, so an occupation will be—it's a little bit broader than a [00:10:00] job title, but it's a sort of a broad class of, you know, jobs that are very related. So you can think about things like an administrative assistant or indeed somebody who works in food preparation. These are examples, you know, of occupations we would have in our data. And to give you examples of transition—so then I'm gonna jump to another study that we did in the Netherlands that had a very similar flavor. But one of the starting points was that when we talked to the Dutch Public Employment Office, they told us, oh, there are many people who want to be administrative assistants.
And so there are, you know, thousands of people, actually, hundreds of thousands of people who apply every year to this type of job, but there are at the moment very few vacancies in that occupation. On the other hand, there are very related occupations, like dental assistants actually, or receptionist, which are, you know, it's not exactly the same, but you can see that the skills and the sort of task you might have that might be associated with these occupations are quite similar. [00:11:00] And we actually do see in the transitions—what our advice is based on—is that we do see people transitioning indeed from administrative assistants to receptionist or to dental assistant. And so we can use that to make recommendations.
Jason Sockin: No, that's great. Sometimes it's just about knowing which jobs there are, right? And actually what am I capable of doing? 'Cause if it's a skill-based thing, then I might not hear about this job title, but might be more than well qualified for another one in the sort of similar occupation. I think in that sort of lens, I think it actually might be nice to dive into job search more broadly and really start talking about these digital tools.
So, you know, we're gonna—you know, the theme of this keynote is matching jobs to people in the 21st century. So, can you tell me sort of what is interesting to you about job search and about matching workers? Why job search, really?
Michèle Belot: Yes. So I'm fascinated by job search. And the reason is that if you think about, you know, the many other [00:12:00] things that we do in life, the many other decisions that we may take in life—like, I don't know, what products we buy in the supermarket or, you know, many of the things we decide on a daily basis—we do a lot. But job search is something that most of us will do a few times. There are other, you know, examples like buying a house or, you know, finding a partner.
But, you know, these are examples of big decisions and big sort of enterprises. And usually many people who are in that space will say that they don't quite know what they are doing. They don't have much information and it makes a lot of sense, right? You don't really have a lot of experience searching for a job. You may not know a lot about what the market looks like. You may not know who else is out there. You may not know what makes sense for you to search, right? So there's a lot of information that is out there, but that information is not necessarily tailored to you, right?
So you might see, well the unemployment rate is high in that region and [00:13:00] low in that other region. But that doesn't mean that you should move there, right? It depends on whether your profile fits with what they're looking for. And so we thought there's actually a lot of information that maybe we could actually try to give to job seekers, and also try to tailor that intervention insofar as possible. And so our goal really—and so I'm part of this really wonderful team at Cornell, which is called Job Search Lab. And our goal is really to try to first study job search, understand exactly what are the barriers on both sides, actually on the side of employers, but also on the side of job seekers. What are the challenges they are facing? And then to try to design tools that really aim at overcoming these barriers.
Jason Sockin: Yeah, I'll just, I'll speak to my own kind of interest as well. 'Cause we both work together and we're both interested in job search and I can say for me, I find it kind of [00:14:00] fascinating because my research sort of centers on this information problem that a lot of things that people care most about are really hard to learn about, right? And so I am really interested in how people sort of peruse job vacancies and figure out what's sort of like between the lines, right?
What's not said, what is said. Even what is said, what does that mean? Is that a promise? Is that cheap talk, right? Is this a place I really wanna work? And so I really like learning about the things that people care about and seeing how people respond when they maybe get some information either from a colleague or a friend about a company.
Maybe they get a referral. And so it's all these things, this information, what do people know about jobs? What do they not know about jobs? And how do they like act on that information? So I think we're both really interested in this sort of informational framework, right? Like what do people know?
What's the potential and where can they kind of apply to. I know that for many job seekers, it can be incredibly frustrating when you don't get the job that you want or you know, you maybe feel like there's a lot of jobs that are sort of like these [00:15:00] ghost jobs that people aren't actually hiring for, and so maybe you get discouraged.
But you know, in terms of a job seeker, what do you think people should be focusing on? The quantity of how many things they're applying to, or really the quality of those applications?
Michèle Belot: Well, I think that's an excellent question and I don't think we have the answer actually. So this is one thing that I would also say is the reason why we went into that space is that we realized that there is actually very little known about what are good search strategies.
You realize that, you know, we don't know. So I wish I could tell people you should do this and that, and so on. And now we have a little bit more idea after, you know, now about a decade of research in this, but there are still many questions that are not answered. And this question of, you know, should you focus on a few employers or actually should you apply more broadly? [00:16:00] I don't think we have a good answer. I guess as an economist I would probably, you know, answer that it probably depends. It depends on the market and what sort of jobs you're looking for. I know that in the economics market we know the more junior professors are actually applying very widely when they go on the job market.
But as you progress, you know, in your career you tend to be more focused. So this is, I think, something that probably applies more generally where when you are younger, you probably will apply to a wide range of places also as part of trying to figure out what you like. And as you said, there's just so much uncertainty about what these jobs are like, what actually suits you, that probably applying more broadly is also a way to learn a lot, learn a lot from interviews and so on. And then figure out what you actually want in a job and then refine your search.
Jason Sockin: No, I agree. I don't think we know this answer and it sort of puzzles me 'cause I could see both sides of it. [00:17:00] Right. And you're trying to like, you know, if you just want a job, maybe quantity, but if you really want it to be a stable job as you kind of researched, maybe quality. And I think, you know, if I had to put my own kind of thought on it, it probably is something that has to do with experience. Right. When you're sort of new into the labor market, maybe you don't know what occupations are right for you. There's a bit of exploring that's worth doing. Right. But I think especially as you sort of fine-tune your skills and get a bit more experience and climb the career ladder, I think quality is probably where you wanna focus, right? Because job search is costly and like transitioning to a job—but then if it's not the right job, leaving—I think you really want to focus on the right match.
Michèle Belot: And actually maybe one thing to say about that is that we also know that because of the new technologies and it's relatively easy to apply now, it's almost like costless, you know, you can just send your CV online and so on. It's not as it used to be where you would have to actually maybe send a letter and so on. So I think that the temptation is there to actually just apply more broadly. But on the other hand, we also know from employers that they then [00:18:00] get flooded with, you know, tons of applications. And that's why I imagine that the thinking of a lot of people is to stand out and to really make employers understand that you are really serious about your application. Then sort of tailoring your application to the employer makes sense.
Jason Sockin: Yeah. We actually have a question from the audience. I think getting into sort of tailoring your resume, right? For sort of what fits. So we have a question, you know, what is your advice about concealing age? I've been surprised, I'm advised not to put years or older experience on a resume. What would be your advice with regards to concealing age or years of experience?
Michèle Belot: So age is a protected category as well. So in principle, you know, employers are not supposed to be using these characteristics to discriminate. On the other hand, it's, you know, inevitable that maybe this is something almost like subconsciously that may be coming up. I think there's a lot of variation also [00:19:00] across countries in what is actually the norm in putting your age or not.
I don't have a good sense of the impact of that. But clearly one should also realize that once you put, you know, your CV, your resume and so on, it's not so difficult to work out what the age is of someone. On the other hand, I think age has a lot of positive effects in many dimensions.
And also we know that recruiters, for example, the two key characteristics they look at are qualifications and experience. And of course experience means how many years you have actually held a job. And that will be inevitably correlated with age. If people are worried about, you know, having been out of the labor force for a long time, for example, there are also other ways, which I think are helpful to try to signal maybe why you were out for so long. Maybe for example, I'm thinking about people who had children and [00:20:00] so on. So this is quite common to also see people putting on, you know, I was on maternity leave or I had two children, and things like that.
Jason Sockin: Yeah, you know, I've heard stories from job seekers that sometimes it's more that they're overqualified for the position they're applying for. So it's not that they shouldn't put their years of experience, just that the years of experience they have or the skills they have maybe are actually over-matching what they're applying to.
And so it could be that the employer sees and goes, this person has actually too many years of experience. So I think it's more just like finding the right match, if you will, like having the right years of experience or skillset for the job that you're applying to. I think when it comes to job search more broadly, Michèle, what tools would you say are out there that are really available to job seekers?
Can you give some examples of what job seekers can rely on?
Michèle Belot: Yeah, so this is, I think, where it's quite exciting times, right? Because things are changing quite quickly and there are lots of technologies that are available. So we've been really interested in how job boards are being [00:21:00] developed and what sort of tools they use to actually try to match employers to people, to workers. There are all sorts of other tools that we have not looked at, but that others in the ILR school are interested in, which have to do with recruitment, you know, trying to actually assess people's skills and so on. So we have been interested in job boards because potentially, I think that they have information from both sides, you know, both from employers and from job seekers. They really intervene at the very beginning of a job search process, of a matching process. And they try to figure out what is going to be a good match? And so we've been particularly interested in recommendation tools.
So these are tools that organize what you are going to see in your feed in a sense. So when you are searching for a job, [00:22:00] you might, you know, say, I'm gonna look for, I don't know, being a professor and then in a particular country. And then I'm going to look at what vacancies are available.
But then I could also see recommendations of other jobs that could be relevant to me that might be other jobs that are really, you know, other professor jobs. But it could also be, I don't know, for me, in my case, it could be things like consultant or economist at a bank or something like that.
And so you would see that. But of course there is a lot of freedom, degrees of freedom in how you organize that, you know, what determines what you see first as a first recommendation and what determines the order. And so we've been interested in what actually works or not. We know, obviously, that this matters a lot, so people tend to click a lot on what they see first. [00:23:00] So it matters a lot for what they look for and what they end up with. So we think it's very important to think that through, and I can tell you more in a second, but—
Jason Sockin: Yeah, no, absolutely. I, and it goes, yeah, recommendation systems are only as good as the recommendations, right? And so part of that is making sure that they actually are plausible if they actually seem like they're being trained on maybe past transitions or skill sets or what may be. But like I said, recommendation systems can be great if they are recommending jobs that make sense. But you know, we've also, I'm sure, seen recommendations where it's just not exactly what we're sort of driving for. I can say that myself too is that I think, you know, I study job search as well and the tools that are just sort of amazing today of this sort of crowdsource information.
And so, you know, I think this kind of gets back into this quantity or quality in that, like if you focus on quality, [00:24:00] right, you can actually put a lot of investment into learning about the jobs you're applying to. Right. And so, you know, I study reputation systems like Glassdoor or Indeed, there's a lot of information out there.
It's costly, right? It takes time to read the reviews, but like a lot of people out there are talking about the insides of companies and I think that just has sort of, you know, revolutionized what we know about being able to get this peek inside. So either doing the research to figure out maybe, you know, what kind of strategies the company is going through, either on their website or, you know, these reputation platforms.
Sort of just learning information maybe that was not present, you know, before the internet sort of revolutionized job search.
You know, we talked about the things that are available for job seekers.
You know, recommendation systems, these reputation platforms, the information. What tools are out there for employers, right? What's available out there for them to make sort of hiring more efficient?
Michèle Belot: So obviously employers are also on these job boards. And then this is what is nice is that sometimes some of these job boards will actually also make recommendations of potential candidates to employers. [00:25:00] So that is a tool. But after that there are indeed other tools that can actually try to, for example, go through resumes in a systematic manner and try to screen out people who might miss certain qualifications, for example, who might actually act as a sort of a pre-screen. We know that there are also tools that try to replace, you know, things like first stage interviews and so on.
And again, I think we are very early in that stage. So I know at the ILR school, we actually talked to HR professionals last year and we were asking them about the use of these tools. And it seemed to me that there was still, you know, some skepticism about the use of this sort of AI-based tools.
But it's true that when one is confronted with, you know, a thousand CVs for a vacancy, one has to somehow screen through those. And these tools can be useful for that. But I think maybe what I think is the worry, and that's what you mentioned about, is that in a way the sky isn't the limit into what one can do to actually try to organize these recommendations. And one worry we have in this space is that, whether it's on the [00:26:00] employer side or the worker side, is that it's very hard to make good recommendations if people don't really know to start with what they are looking for.
And we are particularly worried about recommendation tools that are based on—where the recommendations are based on past searches. And it's something that we will all recognize from platforms, you know, like Netflix, Amazon, and so on, where when you have identified a product, then they will recommend things like that.
And this, you could see how you might want to do that as well in the job market space. But the problem is that if you don't really know what is a good idea to start with, or you have some idea but not a very precise idea, then you might be locked in into [00:27:00] what you have actually searched for initially.
And so we are quite concerned about that, and we are trying to, again, think about ways we can potentially overcome that. So of course you can overcome that to some extent by asking people to give more information about their skills, upload their resume, and so on, and try to have an approach that is more based on indeed the content of their resume and try to match that to the content of the job descriptions.
And so this is another, you know, type of recommendation tools that exist. And so, probably a hybrid model is a better approach. And these hybrid models are used on some of these job boards, but it's certainly, I think not innocuous, what tool is used.
Jason Sockin: It's fascinating 'cause it's kind of this trade-off, right? Of wanting to explore, right? To see what occupations, as you mentioned, are out there. But the more exploring you do, maybe the less fine-tuned that recommendations become because it's learning on your past behavior. Right? So I think how these recommendation systems are sort of what they're trained on and how they sort of adapt are really gonna maybe even lead you down this path where you can't get out of, or these are the [00:28:00] jobs that seem like they're best for you.
Maybe you take the recommendation as advice and then people act upon it. So I do think it's important to figure out what the optimal design of recommendation system is. I can tell you, from one of my co-authors who works at job search data for another platform, one of the most popular searches that people do is blank.
Yeah, they don't, they just look what's out there. And so I think when you have blank, it's really hard to think about what the right recommendation for blank is. And so I think it is really important, but I think that's right. Right. I think you talked earlier about AI and sort of being interwoven into recruitment processes, and I think we've already started seeing this where first stage interviews are chatbots now that can actually go through this conversation.
And I've seen these firsthand and they work pretty remarkably well at having a conversation. You know, they don't seem the most human, but they get the information they need across, right. And so the question is, like, you know, how much do job seekers value talking to a human during the interviewing process?
So I'm curious, you know, speaking about these [00:29:00] tools and these recommendation systems, have people already started to study them? Do we know the impacts of recommendation systems?
Do we know the scope at all about what these are sort of doing to job seekers and to employers?
Michèle Belot: Yes. So I think there's a lot of work in that space at the moment in the US and in different countries. So as I said, I spend a lot of time in Europe. I mean, one advantage that European countries have is that they often have a relatively centralized system.
So that means that if you're a job seeker, you're usually logging in in sort of a central system that sounds very, you know, but it is often the case, which means that you have, you know, data on all the job seekers that may be searching in a particular country. And then you also have very good data on the jobs that they find, the wages that they have and so on.
So that can be really helpful [00:30:00] to understand how these tools work. And there are various teams across Europe who have actually really looked at how these AI-based tools might actually work. And so what we know is that they actually matter, but we also know it's quite hard really to actually make a big difference to people's labor market outcomes.
And I will be a little bit, you know, defending our research. But I think it is true that we think the devil is really in the design, in the detail of how do you design these recommendation tools well, so that they are actually really meaningful for people and can really make a difference.
So we have actually not relied on AI so far for our recommendations. We've relied on algorithms that use past transitions, as I said. But now we are exploring tools that [00:31:00] will use some machine learning and so on. But we are doing this in a very careful manner and trying to alleviate all the sort of potential negative aspects that might be associated with that.
I think ideally what I would like to get to is a job search platform where you can write, you know, almost like an open text and to say, well this is my ideal job, this is what I care about. And then we would be able to actually identify vacancies or jobs that fit what you seem to be looking for. And where one could maybe integrate even information that comes from websites like Glassdoor and so on, that could, you know, actually in a sense, penalize firms that don't have, you know, good reviews on things like workplace culture and so on.
Jason Sockin: Exactly.
Michèle Belot: I think that would be really [00:32:00] nice to have. And this is what, you know, we hope to work on in the future, but there's a little bit of a way to go.
Jason Sockin: If places with bad culture didn't get applicants, they'd have to change their culture, right? And so it's this kind of perpetuating system. So I think that'd be great, Michèle. So I'm looking forward to when you have that recommendation system ready so that our audience can just type in work-life balance and they find the right job for them. I think to close out this job digital tools kind of conversation, is there any way that these tools maybe can be adjusted? I know you talked about it's the how you do the recommendation system that really matters. You know, are there any ways in which we can design them to sort of mitigate any negative impacts that they may have?
Michèle Belot: Yes. So I think that this is again, where there's a lot of research going on at the moment. But, so one team, for example, at MIT has been doing interesting work where what they think about is, for example, to avoid the lock-in effect, you know that you basically keep getting things that look like what you searched for initially. [00:33:00] Of course you can sort of tweak the algorithm so that sometimes it actually goes and explores something that is a little bit out there. Or, as I said, instead of relying too much on your past user searches, you can try to put more weight on other things like the resume of people, the sort of the contents of the resume and how they match to the contents of jobs.
One has to be a little bit careful there, especially when one uses things like, you know, machine learning and so on, is that of course you have to think that a word is a word, and in particular we are worried about sometimes some words being used as a proxy.
So one thing we worry about is that for example, people of certain gender, you know, women might be stuck in certain types of jobs because you know, this was actually a famous problem that was highlighted a few years ago, is that men and women would not see the same kind of ads even if they were searching for the same keywords. [00:34:00] But of course there are things in your resume that even if you didn't put your gender, it might still signal your gender. You know, it might be your hobbies or things like that. And so one has to be careful when one is designing these algorithms that one is really focusing on words that are not like that, and that are things that are really more related to skills and that are really relevant to determine your match to the job.
Jason Sockin: I think that's right. I think it's, again, it's really about what the data is trained on. Right. I know there are some research out there actually studying this, where they're looking at these sort of resumes that they made—this sort of audit study where they're doing resumes and all they're doing is changing the gender and keeping everything the same and the recommendations that these, you know, ChatGPT, these large language models are giving are actually quite different, right. And even recommending a pay gap in that. And I think that's just sort of reflecting data that's sort of out there. And so it really depends upon, you know, what's being trained on.
I think you said sort of mitigate these negative effects. I wanna go to a question from the audience, [00:35:00] which is, uh, Michèle, do you think using AI to prepare for interviews is a good idea or not?
Michèle Belot: This is again a very good question. I mean, I think any preparation for interview is probably, you know, a good idea. And I think it is true as we have all seen and experienced that these tools are pretty good and are getting better by the day. Uh, so having someone with, or someone or having an AI agent that's never gonna be tired of, you know, you rehearsing the same questions and asking you the same questions and giving you feedback.
I think there's a lot of value in that**. Of course there's a lot of also, uh,** variance in how, uh, how good these tools are. But I, I actually think it makes**—**it makes sense to me.
Jason Sockin: I'm, I'm gonna say yes, but caveat in terms of my answer**,** 'cause I'm actually, I'm doing a project on interviews right now, so I'm trying to learn all the different ways in which firms should try to screen or how workers sort of learn from interviews.
I [00:36:00] can tell you, I was at a workshop recently. I saw a paper**—it wasn't about interviews, but it was about AI tutors,** right**?** And so like, they basically had students that could have like this AI tutor that trained them exactly in the material for the class. They had an AI option that was not a tutor**—t**hat was just basically would give you the answers.
And then they had a section that didn't have the AI, and the AI with the tutor actually did better. The AI without the tutor did not. And so I think when you're using AI to prepare for interviews, I think it's about tutoring, right? You can get the answer, but you have to sort of learn and sort of engage in it.
But if you just sort of memorize the answers, I don't know if that's gonna be helpful**,** right**?** I think, you know, there is a lot of information out there on interviews. Again, like some people describe their interview experiences, so depending on the types of jobs**—and of course not every job has information out there—**uh, but for some jobs people describe the interviews they went to.
And you'll see like practice questions, right? Or like**,** you know, when it comes to consulting or cases, software engineering, there's practice questions. I think if you look into it, thereare like materials out there and I think using AI to really learn can actually be incredibly helpful**,** [00:37:00] right? Yeah**,** because now those answers can be provided, but you know, you have to also learn from those answers, I would say.
Michèle Belot: Yeah. And one, one additional thing I would say is that it's maybe obvious, but of course**,** you know, all these AI uh, recommendations or these, these things that you get by typing something into ChatGPT or other tools, they are obviously**,** you know, they're predicting, you know, what would be a sensible answer based on data that exist.
And so that means that you are likely to look like the average person who is interviewing**,** right**?** For example, if you are asking for help on how should I answer that question**?** Well, these models will actually predict what makes sense most of the time based on the data they have. And so these are not great tools to help you make, you know, stand out**,** for example.
Even though they could maybe actually give you as a tip that you have to come up with something to stand out. But I think this is where, [00:38:00] again, maybe this is another caveat**—is that if you really want to stand out, these tools have—they're limited** as well, because you're not gonna stand out by just using their sort of ready-made answers.
Jason Sockin: Yeah, no, absolutely. I think speaking of standing out and sort of open-mindedness, if you will, getting back to the theme from the beginning, I think it**'d** be important**—So we have about 15 minutes left for our discussion. I'd love to learn more about the Job Search Lab. I know you have this lab that really is sort of tailor-made to help us better understand how people search for jobs. Can you tell me and the audience more about that lab? What kind of research, what kind of initiatives,** are you really doing and studying there?
Michèle Belot: Yeah, so this is really great. So this has been, uh, an initiative that we started when we joined the ILR School. So, uh, Philipp Kircher, who joined at the same time as I did, is also an expert in the sort of the job search and matching process. And we actually were really fortunate to find an amazing postdoc researcher who has been working with us for, uh, a number of years now.
Uh, and together with also another wonderful graduate student, Via, they, they have been working on developing a job search platform that is meant to be used for, um, for real. So to really help people find jobs, but it is also meant for, for us to use as a research tool. So what we really want to do with that is this will be looking like a, a normal, you know, job search platform.
We have actually access to thousands of vacancies, like, you know, many other, uh, job boards. At the moment, we've been focusing on New York State because this is where we are located and it's also really, uh, great for us to potentially, uh, indeed look at**,** you know, vacancies and to locate job seekers who are in the area.
And so the goal is really to, really [00:40:00] experiment or test different tools that we think could be helpful and to see if they are actually helpful. So when people join the, the job search platform, which they can do anytime**—can do that uh, today if they want to, um, they can actually participate in a study and there, uh, what we do is that you will be actually getting access to the job search platform for free. And you can, you know, navigate as you would normally do.** But then there might also be sort of additional things that, additional features that we implement and that we want to test**.** And also we will ask you if you participate in the study, we ask you as well to, uh**,** answer some, some surveys. So that will be helpful for us to understand how is your job search going? Are you actually finding jobs? What kind of jobs are you finding? Are you happy with your jobs? And we even compensate you for actually answering these, uh, these surveys. But so for us, it's really the goal to test what we think [00:41:00] are tools that are frontier tools to help you navigate the labor market and to really get feedback from, uh, job seekers about their experience and whether this is helpful or not.
Jason Sockin: Yeah**,** Michèle, can you tell us a bit more**—like, why, why is it so important to have a lab to study,** job search? As you said, like there are platforms out there. There are currently people looking for jobs elsewhere. You know, what can we learn from this lab that maybe you can't learn anywhere out in the world? Just by talking with folks and asking them, what can we really learn from this lab design?
Michèle Belot: Yeah, so I think one learns a lot by talking to people as well. So we, we really like to do that as well, to talk to job seekers and to talk to people who are actually operating these platforms.
It is true that what is nice here is that if you think about. You know that these big job boards, they might have different objectives, right? I don't know what their objectives might be. It might be to maximize the number of clicks or maximize, you know, something that we don't know. We are [00:42:00] really interested in, you know, what happens to people, whether they actually find jobs, how happy they are with their jobs and so on.
This is usually data that is very, very difficult to obtain, even if you are good at collecting data and some of these job BO boards are. Usually when you found a job, you know, think about LinkedIn or something. You might actually write, yes, I found a job and this is my new job, but you will not report how happy you were with that job or even information on wage and so on will not be available.
So we think there is a lot of use and value in actually collecting additional information so that we can guide other people as well towards jobs that may, uh, may make sense. And also, uh, maybe this is more the sort of academic perspective, is to really understand how these tools work for people and whether they are helpful.
To them or not, we really need to [00:43:00] be able to tweak that ourselves. So we want to really test one thing at a time. And we really want to have these comparison groups know that some people, not everybody's getting the same features at the same time. And that's very helpful for us to really understand what works and what doesn't work.
Jason Sockin: No. Yeah, absolutely. I, I think it's, it's one of those things where when you're out in the world, everything is changing at once, but when you're in a lab, you can really figure out what features, what designs, what parts of these are the recommendation systems or part of the platforms actually, you know, improve or do not improve workers' outcomes.
And so I think it's really about learning and trials and trying to figure out how to like design the best platform in the end. I know you are currently running a large study at the lab. Can you tell us a bit more about it?
Michèle Belot: Yes. So we are running, uh, uh, at the moment, a large study. We're hoping to recruit up to, you know, 1500 job seekers in, uh, New York State.
So if you are a job seeker, uh, at the moment and you are located in [00:44:00] New York State, please reach out it's easy to find is the Cornell job search uh, lab. The goal is really to make our job search platform accessible, widely accessible.
So you can search as you, you'd normally do on a, on a different job search platform using keywords, having a preferred location and things like that. but then again, we will also, uh, try to test different tools, different ways of recommending jobs in particular. We, we want to also vary the, uh, the information that people might have about different aspects of the labor market.
because again, there's so, there's so much that one could give people information about, You know about where, uh, where there are more jobs or, uh, what employers are looking for and so there are many pieces of information one could give and we want to see what is actually valuable, what seems to be useful to people in their search.
Jason Sockin: Hmm, that's great. Yeah, I think that 1500 job seekers sort of study, I'm sure it's gonna be super [00:45:00] impactful
Chris Wofford: Thank you for listening to Cornell Keynotes. If you're interested in learning more about the Cornell Job Search Lab and participating in the research, please be sure to check the episode notes for details on how you can become part of this. So, I want to thank you for listening to Cornell Keynotes, and please subscribe to Stay in Touch.