An interview with Patrick Hall, Principal Scientist at bnh.ai and Advisor to H2O.ai

An interview with Patrick Hall, Principal Scientist at bnh.ai, Advisor to H2O.ai

Sayak: Hi Patrick! Thank you for doing this interview. It’s a pleasure to have you here today.

  • ML can break the law, get a company sued, or get you or your organization called-out in the press, today. If you’d like to avoid these situations in the future or need help now, we are here for you! I also think we can expect more ML-specific regulations in the future and we can help organizations get ready for those as well.
  • I also advise H2O on how to incorporate ideas around explainable AI (XAI), interpretable models, discrimination testing and remediation, data and ML privacy, security, and model governance into their ML software products.
  • My motivations became more human-centered later, when I got to H2O, I learned how impactful AI is to people, and that everyone would want to understand this tech.
  • Now my motivations are centered around human needs and risk. ML is great but has risks associated with it, like nearly every other technology. Interpretability is key to understanding risks. You can’t mitigate risks you don’t know about. These risks can cause people harm and prevent organizations from using ML. With bnh.ai, I want to help data scientists and organizations innovate with ML in a responsible, risk-aware way.
  • It was really difficult and still is very difficult to communicate complex ideas about how ML models work to non-technical professionals. Through the years at H2O, we learned to simplify, simplify, simplify our user interactions. However, I think all software developers still have a long way to go in explaining explainable AI to users through software.
  • Regulatory compliance was hard in the beginning and also continues to be a pain point. The people who need transparency the most are often in regulated industries and they need real answers. Half-baked, approximate solutions are not interesting to them. This is another reason why I started bnh.ai actually. Compliance questions are extremely difficult, and cannot be solved by data scientists alone. Data scientists need legal help for compliance use cases, and you can expect ML to become more regulated over time, so more use cases will require compliance.
  • This includes getting domain experts, social scientists, and legal, compliance, risk, audit colleagues involved from the beginning. Sadly, I’m seeing a lot of data scientists ignore this advice right now with COVID-19. Very few people out there are qualified to do this kind of epidemiological modeling and data analysis. Honestly, I’m pretty disheartened by “armchair” or even commercial ML attempts to capitalize on this tragedy, especially being so arrogant as to think this is an ML problem and data science can solve alone, without input from medical domain experts, ethicists, and policy experts who understand applicable regulations in the healthcare and potential public health outcomes.
  • Another reason you must consider a holistic approach is because accuracy, fairness, interpretability, privacy, and security are all linked. Consider for instance a model deemed fair by some definition at training time, that is later hacked to become discriminatory, or think about trying to debug an ML black-box that is leaking sensitive private information. You can rarely have one of these attributes without the others in my opinion.
  • Such a holistic approach also includes considering AI incident response. How will you respond when your AI fails?
  • I hope for more and better representations of causality in model architectures and more real-world ability to conduct causal inference.
  • I expect more types of directly interpretable models for unstructured data, and less reliance on pure black-box deep learning, e.g., this looks like that deep learning models.
  • I’d like to see more interpretable deep learning models for structured data, e.g. explainable neural networks (XNN).
  • I’d also hope for the continued maturation of model debugging methods and practices, e.g. Debugging Machine Learning Models or Real-World Strategies for Model Debugging.
  • I also know I am a VERY traditional “book learner.” People learn in different ways. So I can only say what works for me. I learned about ML through textbooks and FORTRAN. (I’m probably in the last generation to do so.) I’d actually turn this question around to you, Sayak, to hear what people do these days.
  • These deep learning models, aside from being unintelligible black-boxes, were simply not deployable in any meaningful way. Very few, if any, enterprise database systems or scoring engines could run Python at that time, much less CUDA. Moreover, results were being reported on static, well-known, benchmark datasets, and not on live, new, unseen data. I’ll always remember this example of how the hottest thing on the web was near meaningless on the ground and I ended up reflecting on this experience a few years later in The Preoccupation with Test Error in Applied Machine Learning.
  • Try to follow the scientific method. Form a hypothesis then conduct an experiment.
  • I only know because I do this myself, but data scientists often fall prey to confirmation bias. We snoop around a data set, talk to our colleagues, then devise a hypothesis to test through a modeling experiment. That’s not following the scientific method, and you’re likely to find what you or your colleagues thought would be in the data, whether it exists in reality or not because confirmation bias is such a strong type of bias.
  • Personally, I interpret the scientific method as forming a hypothesis, collecting appropriate data, performing an experiment, and then analyzing the results to determine if the hypothesis is true. I’m really not sure how combing through redundant, noisy, incomplete data, collected for some operational purpose (i.e., “data exhaust”), looking for a hypothesis to test, and confirming that hypothesis with an overfit black-box model came to be called data “science.”
  • All of us, data scientists, should learn more about experimental design techniques.
  • Engage with the communities in which you are interested politely and professionally. Ask them questions.
  • Be aware of history:

Fifty Years of Data Science

50 Years of Test (Un)fairness: Lessons for Machine Learning

Statistical Modeling: The Two Cultures

A Very Short History of Data Science

  • Treat Kaggle as a learning platform, not a religion. Kaggle is a game. Real-world ML is not. And if you treat real-world ML like a game, you could do more harm than good.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store