The Erdös Institute Deep Learning Boot Camp is a selective program designed as a follow up to the Data Science Boot Camp (see below). Projects are evaluated by the course coordinator as well as industry partners.

The parameters of this project were provided by industry partners at Aware, who played the role of primary stakeholders throughout the project. They provided a real world dataset (a massive collection of Reddit posts), guidance, and feedback throughout the project.

objectives

Implement the retrieval component of a RAG pipeline such that it is able to:

• Retrieve highly relevant results
• Retrieve results quickly (less than a second)

methods

• Processed and stored documents in a database using Polars and LanceDB
• Used existing metadata to impose some structure on the data, and engineered new metadata that leveraged the inherent relationships between documents.
• Implemented language embeddings to build a vector database, and then an ANN Index, to speed up queries using LangChain, HuggingFace, and LanceDB.
• Developed/Adapted three metrics (Mean Reciprocal Rank, Extended Mean Reciprocal Rank, and Normalized Discounted Cumulative Gain) for evaluating model performance and improvement, and manually labeled a set of sample queries and results to establish a baseline.
• Tested more than 150 retrieval configurations, varying pre-embedding parameters such as chunk size and embedding model, as well as pre-retrieval filters implemented using SQL queries, and then re-ranking using engineered metadata.

results

• Achieved significant improvement over baseline across all three metrics, including a 40% improvement in Extended Mean Reciprocal Rank over baseline configurations.
• In the best performing configurations, the top result was 2X as likely to be relevant to the query (as opposed to just related, or not relevant).
• Retrieval times we kept below ~300ms when querying the entire dataset (more than 5.5 million documents).

The Erdös Institute Data Science Boot Camp is a selective program designed to prepare PhD holders, and PhD students, for careers in data science and machine learning. Projects are evaluated and ranked by industry experts from a variety of disciplines.

objectives

• Build a model that is able to forecast groundwater levels using weather and surface water data.
• Present results to stakeholders in a format that is interactive and easy to utilize.

methods

• Collected, cleaned, and compiled data from a variety of government and commercial sources.
• Selected and engineered features for model training.
• Trained and evaluated several supervised machine learning models
• Wrapped keras models in custom scikit-learn transformers/estimators to improve training and tuning efficiency
• Selected and fine tuned a convolutional neural network (CNN) model, using a long short term memory (LSTM) framework, for our final model

results

• Our final model produced a robust forecast that was capable or making accurate (within ~10%) predictions as far as 7 years beyond the training data, a 67% improvement over baseline models.
• Results were summarized and made available via an interactive streamlit web app.

This certification includes courses on topics such as:

• Building ML models with NumPy and scikit-learn, building and training supervised models for prediction & binary classification tasks (linear, logistic regression).
• Building and training neural networks with TensorFlow to perform multi-class classification, and building and using decision trees and tree ensemble methods
• Applying best practices for ML development, and using unsupervised learning techniques for unsupervised learning including clustering and anomaly detection
• Building recommender systems with a collaborative filtering approach and content-based deep learning methods, and building deep reinforcement learning models

While earning this certificate, I earned several supporting certificates in the following:

• Supervised Machine Learning: Regression and Classification - view certificate
• Advanced Learning Algorithms - view certificate
• Unsupervised Learning, Recommenders, Reinforcement Learning - view certificate