Experience

(May 2024 - Present)

Working under Prof. Eugenio Culurciello in e-Lab to develop an agent that can help users by answering questions or performing tasks given by them in indoor environments.
Conducting research on Large Visual Models, including fine-tuning, prompting, and instruction tuning to achieve optimal model performance.
It has various applications such as enabling the robot to remember objects it has seen and inform users of their locations. Additionally, it enhances robot navigation capabilities, enabling the robot to perform tasks such as moving to the kitchen and doing dishes upon user request.
Skills: Python, PyTorch, Transformers, Langchain, Numpy, LLMs, LVMs, Multi-Modal Models, Fine-tuning, Instruction tuning, Quantization

(Sep 2023 - Sep 2024)

Conduct research on multi-robot long horizon planning using the Monte Carlo Tree Search (MCTS) algorithm to efficiently rearrange objects into complex desired goal locations.
Ensured collision-free navigation for the multiple mobile robots all the while aiming to achieve a highly efficient plan that can produce interesting emerging behaviors for the robots reducing the overall distance travelled.
Skills: Python, PyBullet, Numpy, RRT star, Monte Carlo Tree Search, Cost Based Search Due to ongoing research, I am unable to share the current images and videos of the research. The above images are a simple version of expected results I am working on. Source

(Apr 2021 - Jun 2023) - 2yrs 3mos

Worked on the agent-based natural language support for the WSO2 Mobile app. This feature enables users to interact with the WSO2 Mobile app using natural language. The agent is capable of understanding the user’s intent and perform the required actions.
For example, the user can ask the agent via voice or text to “Show me the sales report for the last month” and the agent will understand the user’s intent and show the sales report for the last month.
The Ballerina Agent library uses the OpenAI GPT3 API to understand the user’s intent and then it invokes the REST APIs with the required parameters extracted from the user’s intent to perform the required actions and return the results to the user in a human-readable conversational format to provide a seemless chat experience.
Skills: Ballerina, LLMs, OpenAI GPT3 APIs, Azure OpenAI, React, OAuth2, Dockerization, Cloud Deployment, Full stack development.

Co-developed a GPT-3 Davinci powered chat assistant which uses WSO2 Choreo Documentation as a knowledge base to answer user queries and is also capable of citing the sources and grounding the answer based on facts and avoiding hallucinations.
Used a Vector Database called Weaviate to store the documents as small chunks and used Prompt Engineering to guide the models in giving a conversational and factual chat experience.
Skills: Python, OpenAI GPT3 APIs, Weaviate, Vector Database, Prompt Engineering, Chatbot, Full stack development, React, OAuth2, Dockerization, Cloud Deployment

Used the Ballerina Agent library to tryout the numerous use cases and to understand the capabilities of the library using multiple internal APIs of WSO2.
The Agent library is an improved version of Langchain and it focuses on seemless API invocations using natural language.
I have closely worked with improving the library to support the use cases of WSO2 APIs.
Skills: Ballerina, LLMs, OpenAI GPT3 APIs, Azure OpenAI

Solely responsible for the entire ML lifecycle of “Next low code suggestions” feature for a WSO2 product called Choreo where I worked on data collection, preprocessing, feature extraction, model training, deployment all the way to talking with relevant stake holders, coming up with highly scalable architecture design, project planning and gantt charts.
Developed a parser that can extract relevant information from the raw data comprising of Ballerina Syntax Trees from the users’ code and applied various feature engineering techniques to extract valuable features from the data.
Explored various ways to utilize the data by modelling as a sequence to sequence problem and tried out various deep learning models like LSTMs and LLMs like GPT-3 and GPT3.5 to predict the next possible low code element that the user might be interested in adding.
Skills: Python, Keras, TensorFlow, Deep Learning, Natural Language Processing, LSTMs, LLMs, GPT3, Data Preprocessing, Feature Engineering, Data Visualization, Data Analysis, Data Collection, Data Labeling, Data Augmentation, Data Cleaning, Data Wrangling, Exploratory Data Analysis (EDA), Data Transformation, Data Storage, Unit Testing, Integration Testing, Model Training, Model Evaluation, Model Deployment

This is a continuation of the previous project where I worked on developing the entire backbone for the low code suggestion feature.
Designed and developed pipelines to automate feature extraction, model training, model evaluation, model deployment and model serving.
This helped to automatically collect new data and rapidly iterate through the entire ML lifecycle and to deploy the best model to production in a matter of hours.
Skills: MLOps, Python, Azure ML Studio, Keras, Tensorflow, Azure Table Storage, Azure Blob Storage, Azure Data Lake, Azure Machine Learning, Azure Data Explorer

(June 2019 - Dec 2019) - 6 mos

Experimented with the Gaussian Process Bayesian optimization model improving the 99 percentile latencies of the microservices in realtime by using the real-time load configurations and feeding them into the model.
Skills: Python, Gaussian Process Bayesian optimization, Ballerina, Microservices, REST APIs, SSH, JVisualVM, JMeter, Load Testing, Performance Testing