Continuing the discussion from the Building Effective Data Science Teams Webinar:
Delivering Real Value
Data scientists sometimes have a reputation for valuing their autonomy or wanting to spend their time on fancy algorithms over delivering valuable results. Do you think that’s fair? How do you deal with this?
Our panelists for this webinar were:
- Kobi Abayomi, Senior VP of Data Science at Warner Music Group
- Gregory Berg, VP of Data Science at Caliber Home Loans
- Elaine McVey, VP of Data Science at The Looma Project
- Jacqueline Nolis, Head of Data Science at Saturn Cloud
- Nasir Uddin, Director of Strategy & Inspirational Analytics at T-Mobile
- Moderated by Julia Silge, Software Engineer at RStudio, PBC
Some discussion in the webinar:
Nasir : Most data scientists would like to leverage the latest and greatest types of algorithms, like deep learning. In reality, you may be able to solve the business problem with simple regression techniques. As we’ve mentioned, it’s about building credibility and delivering transparent results. It’s not about the latest and greatest technology, it’s more about how you can deliver meaningful outcomes to the business and that you can explain it. Whatever you’ll be delivering, you can explain it better.
Model explainability is a huge field and there is still a lack of explanation of very complex black box types of models. Data scientists should not go to complex techniques first but try simple techniques, and see whether you can answer the business question.
Jacqueline : I do think it’s very fair. I think we, as a field, set this up. What’s every big blog post? “Check out GPT-4! Look how much bigger and better - we make the exciting stuff, the new giant GPUs, whatever.” I think when junior data scientists come in they have a zest for trying the latest and greatest, which is okay but you’ve got to have a sit down conversation and explain that the metric isn’t if the model the most technically accurate. Is it easy to use? Is it robust to rerun?
As a junior data scientist, I learned the hard way when a bunch of my work had to be thrown out. If you have senior data scientists still making this mistake later in their career, now that’s a problem. That’s a toxic scenario where you have someone who has a lot of influence and power not helping out the business. I would say as the manager, it’s your job to assess how much can I guide them towards the right path and how much do I have to say “this behavior is actually going to cause major problems and we need to have harsh talks and cut it down?” I do think that’s a very real thing that happens all the time.
Julia : I agree with both of you that this is real - this is a stereotype because it’s real. It’s something for us, as a field, to grapple with as we think about how we are going to be effective in organizations.
Elaine : Building on something Jacqueline said before about companies hiring data scientists just because they have data and the goal isn’t clear. I think we can set ourselves up to be more successful here, partly in the hiring process, by not lying to people and saying we’re doing these cool, new, sophisticated things. I try to be honest with candidates about the places where we might get to do really cool new things. There’s also a lot of getting the right data, providing averages, and doing some of that data democratization. If what you really want to do is do the latest and greatest all day long, you need to go work somewhere where there’s real business value in that bleeding edge. At most places, that is not what we’re doing the vast majority of the time and can potentially weed out some people who just don’t understand what they’re getting into.