Making the most of data science teams
Lessons learnt as a Tech Lead
There is a wide variety of structures for data science teams, but given their specialisations data scientists often work in cross-functional teams. Cross-functional teams consist of people from different job functions within an organisation (e.g. software engineers, data scientists and scrum masters).
A cross-functional team is ideally responsible for an entire value stream, which brings the benefits of being able to work with a high degree of autonomy and velocity. For example, a cross-functional team could own an entire predictive maintenance solution including data pipeline, machine learning model and user interface.
That being said, it can be tricky to make the most of cross-functional teams. It’s more difficult to facilitate knowledge-sharing across teams and it can be challenging to align different ways of working.
With some years of tech lead experience I would like to share my opinions on what makes a great data science tech lead — there’s a lot more than just the technical skills.
1. Understand the skills in the team
A common challenge for leaders of cross-functional teams is that they tend to be stronger in a specific functional area. For example, with my background as a data scientist my strengths are in the design and prototyping of models rather than software engineering and deployment. This can easily lead to biases and wrong assumptions when planning and setting goals.
To maximise the potential of a team it is essential for any leader to understand the individual motivations, skills and aspirations of every team member. In tech companies this is typically associated with management roles. However, in my opinion if you turn a blind eye to the needs and interests of your team members you won’t fully leverage their skills and potential.
I would recommend checking in regularly with team members and asking questions such as
- In our current project, what are you most excited about working on? Generally, what kind of work excites you?
- What are the skills you want to grow?
2. No cross-functional team without crossing functions
Just having team members of different functions in a team doesn’t make it cross-functional. If you stick to rigid role definitions you will end up with skill/responsibility gaps between roles.
In my experience it is essential for the team leader to identify such gaps and grow the overlap between functions to address them. For example, an ML engineer should have a basic understanding of the models the team aim to run in their product. These interactions also help ML scientists understand non-functional requirements (such as runtime performance) early on.
I think the best way to cross functions in a team is to set outcome-oriented goals that span multiple functions and not only encourage, but require collaboration.
Examples of outcome-oriented goals:
- “Predictions from a placeholder model can be seen in the user interface”
- “Model maintains prediction accuracy applied to a new (similar) dataset”
Another great way to fill the gaps is to find team members who want to expand beyond their core skills (see point 1).
3. Extreme ownership and trust
“The leader must own everything in his or her world. There is no one else to blame. The leader must acknowledge mistakes and admit failures, take ownership of them, and develop a plan to win.” from Extreme Ownership by Jocko Willink and Leif Babin
Applied to a data science team the “extreme ownership approach” means that the leader should take responsibility for the team, the whole solution and its delivery. If you don’t have a background in, say, DevOPS and there is a problem with the CI/CD pipeline slowing down the team, you can’t just blame your engineers. If team members are burnt out don’t just wait for their line managers to sort it out (or even worse, for them to leave).
Any leader should see it as their responsibility to jump on a problem that’s affecting the team or the delivery — even if it’s not part of their role description.
Importantly, extreme ownership doesn’t mean micro-management! The key to using extreme ownership as a tech lead is to build trust with team members. With a good understanding of team members’ skills you should be able to anticipate how much support they need when working on a task or project. Additionally, they should also have the psychological safety to bring up and talk about uncomfortable issues or to ask you for help when struggling with a task.
4. Expand beyond your core skills
Tech leads typically own the technical decisions in the team, set the technical direction and represent the needs of the team to the wider organisation. If you are in such a role (rather than a management role) you most likely have to grow your technical skills beyond your core capabilities to meet these responsibilities.
Of course, you shouldn’t expect to have the same technical depth as your more specialised colleagues. However, if you don’t expand your skillset you can become more of a coordinator than a leader; setting up meetings and connecting colleagues together rather than actively contributing.
It’s often difficult to carve out the time for learning and development. The typical advice is to break personal development goals into manageable chunks. In the past it’s helped me to do the following:
- To improve theoretical understanding: Ask colleagues for resources such as books, blogs, online lectures
- To improve your code review skills: Learn from senior colleagues. When they review a pull request, what do they comment on and why?
- To learn about a codebase: Add small bug fixes or features to unfamiliar code repositories
Conclusion
It’s tempting to focus on the technical responsibilities of a tech lead role, but hopefully this writeup highlights some other aspects of the role: understanding the needs of your colleagues, broadening your skills and developing a healthy work culture in your team.
