Agenda

Machine Learning Week Founder Eric Siegel will welcome you to the event and kick things off.

At Paychex, we used large language models like SBERT to match client inputted job titles to a taxonomy of 800+ job categories provided by the US Bureau of Labor Statistics. Our multi-level architecture combines SBERT with neural network classifiers to achieve high matching accuracy. The SBERT model was fine-tuned in a Siamese network approach to improve its performance. The product will be available to our clients to recommend the best matching BLS codes for their inputted job titles. Normalizing job titles will provide Paychex clients with advanced wage or retention analytics.

Given ML is here to stay, and growing in complexity by the day, it has become now critical that we think deeply about how to properly align our models with human interests. Doing this correctly requires a cultural shift in the way most organizations approach the model development process. Traditional metrics like F1, accuracy, or ROC on held-out test sets are woefully inadequate indicators of alignment when considered on their own. However, with the right tooling and a clear alignment framework, if we apply the principles of TDD (test-driven development) in a collaborative and transparent manner to ML development, we can not only build more performant models with greater velocity, but also instill far greater confidence in the models we are integrating into human lives.

While there is a lot of talk about the need to train AI models that are safe, robust, unbiased, and equitable - few tools have been available to data scientists to meet these goals. This session describes new open-source libraries & tools that address three aspects of Responsible AI. The first is automatically measuring a model's bias towards a specific gender, age group, or ethnicity. The second is measuring for labeling errors - i.e. mistakes, noise, or intentional errors in training data. The third is measuring how fragile a model is to minor changes in the data or questions fed to it. Best practices and tools for automatically correcting some of these issues will be presented as well, along with real-world examples of projects that have put these tools for use, focused on the medical domain where the human cost of unsafe models can be unacceptably high.

Product recommendation is at the heart of Personalization group’s efforts to help Albertsons customers. Deep Learning has become the go-to approach for recommendation. Therefore, the group has begun to put efforts into applying Deep Learning to enhance new product recommendations. First, leveraging transaction data and product catalog, we built Customer DNA and Product DNA models. Customer DNA model captures customer characteristics such as purchase behavioral pattern, dietary preference, e-com affinity, customer location, etc. and embeds into a list (vector) of numbers. Similarly, Product DNA model captures product characteristics (e.g., is product organic and/or sugar-free?) and product-product associations—e.g., bread and peanut butter are usually purchased together. Second, we leverage these models to build a next generation recommendation system—inspired by the Wide and Deep recommendation model architecture. Our experiments building the framework have generated favorable results and we will share our journey from model conception to putting it in in production to better serve our customers.

Document AI is one of the Machine Learning fields that study about understanding and analyzing documents like PDFs, DOCXs, etc. But with the growth of complex documents and limited data due to data privacy and security, industry ML practitioners are looking for a method that could augment data effectively. At Otrafy, we developed an augmented method that is a combination of CV and NLP where we augmented data based on both text and layout of the document. The study show that our works improve the performance of LayoutLM in both accuracy and runtime.

At Northwestern we have developed a system that is built to consume and capitalize on IoT infrastructure by ingesting device data and employing modern machine learning approaches to infer the status of various components of the IoT system. It is built on several open source components with state-of-the-art artificial intelligence. We will discuss its distinguishing features of being Kubernetes native and by employing our work that enables features to be specified through a flexible logic which is propagated throughout the architectural components. We will discuss select use cases implemented in the platform and the underlying benefits. The audience will learn how to build or use a streaming solution based on Kubernetes and open source components.

In fintech, it's important for companies manage their credit risk because if customers don't repay their credit, the lender loses money. In this talk, we'll explore how to prevent credit risk using a neural network model. We'll discuss not only what worked, but also what didn't work, and how the building blocks of the full machine learning system. We start with one model and discuss how to layer on more of an ensemble approach to predict risky customers and take action in a way that doesn't cause them to become even more risky business. We'll discuss features, a neural network model, evaluation, serving, monitoring, and ideas to improve.

The conference chair will provide remarks and kick off Day 2 of the main program.

At the RealReal we take in over 10k items daily. For each item 3-6 images are captured. Historically those images would be "retouched" by hand which include tasks such as aligning the mannequin, cropping at one of three fixed distances based on the length of the item, digitally removing the base of the mannequin and its shoulder seams all while providing a "luxury" feel. We've learned lots along the way to automating 87% of our supply saving millions each year.

In this talk, we will talk through the journey of building an end-to-end forecasting platform with a focus on feature engineering, consolidation of features in a single place (feature store), and leveraging DNN techniques to solve different forecasting problems like Demand forecasting, Replenishment, Inventory optimization, etc. powered through the same platform

As part of the session, we will review the journey with the following stages :

• Feature Engineering: Experimentation to add critical features like Product Hierarchies, Price Rank Ratios, historical aggregated features, etc. along with their impact on the model accuracy
• MLOps : Use open-source Kubeflow to orchestrate and automate the data load, feature engineering, and Model training along with Model Deployment
• Decomposition of Forecast: Decompose forecast into Base & Promo forecast for What-if Simulation to identify the impact of Promos on Forecasts. Also, expanding the approach to inventory optimization.

Language models have revolutionized the field of Natural Language Processing (NLP), and large language models in particular have opened up new avenues for research and applications. In this session, we will explore the key features and underlying mechanisms of large language models, and discuss their implications for NLP research, industry, and society.We will begin by introducing the basic building blocks of large language models and discuss the challenges of training and evaluating large language models, as well as their limitations and potential biases. We will then explore some of the key applications of large language models and the impact of large language models on industries such as healthcare, finance, and entertainment, as well as their potential for use in improving accessibility to information and services for marginalized communities.By the end of this session, participants will have a deeper understanding of the opportunities and challenges of large language models, and a broader awareness of their impact on NLP research, industry, and society.

Large language models are taking the machine learning industry by storm. Amid an outpouring of new research and exciting updates is the business question of how we can actually take advantage of this cutting-edge technology in the business world. Glean, an enterprise search engine that helps people find what they need and unearth things they should know across all their apps, is one of the only companies in the enterprise world which leverages LLMs jointly with enterprise data. We'll talk about lessons learnt from doing so, and how can enterprise companies build AI teams from scratch.

The surging demand for Natural Language Processing (NLP) and Large Language Models (LLM) has broken glass ceilings across industries in recent years. In fact, as per Fortune Business Insights, the global NLP market is projected to reach $161.81 billion by 2029, suggesting its importance to businesses and the broader economy as the volume of unstructured data increases. 

As we become more reliant on NLP in our daily lives, it's crucial to be aware of the potential risks that come with it. In the absence of monitoring, these models are prone to causing unintended consequences and posing significant risks to businesses, from reputational damage to regulatory non-compliance.

Even the best human minds need sharpening, and the same goes for complex NLP models. Implementing embedding monitoring practices can help detect and reduce drift and bias, understand black-box model decisions, improve model performance, and minimize technical debt.

In this session, Devanshi Vyas, Co-founder at Censius will deep dive into the implications of unmonitored NLP and LLM in today’s world. Explore the key components of AI Observability, including Monitoring and Explainability, and how they can be applied to drive positive business outcomes.

Key takeaways: 

• Explore the current state of unstructured data models possessing massive complexities
• The need for Observability in NLP to improve visibility and curb AI risks
• Strategies to leverage AI Observability with embedding monitoring to proactively troubleshoot models for optimal performance

In this talk, I will go over the common machine learning system design principles in speech models.Firstly we will discuss, when Natural Language Processing (NLP) models have to be stacked after speech recognition models, the common error recovery mechanisms followed for a large scale prod system.If the model has to tackle growing use-cases like learning new key words, how the core model has to designed to accommodate new use-cases without retraining the entire DNN model. In this talk, the audience will learn that in-practice speech models and NLP compliment each other for better recognition rate.