Enhancing the customer experience for a Top-10 tennis tournament with an advanced AI chatbot

Partner Overview

In partnership with AWS, Neurons Lab supported one of the world’s biggest tennis tournaments on the men’s and women’s professional tours.

The tournament organizer needed to improve digital engagement and the customer experience with a generative AI-powered chatbot, requiring a partner able to:

• Build a chatbot that uses structured tournament data, answers questions quickly, and maintains a conversation via WhatsApp.
• Organize knowledge about the tournament based on its website and closed information sources as a vector database ready for semantic search and retrieval.

Business Challenges

With the project beginning only six weeks before the tournament itself started, the delivery deadline for a ready-to-launch chatbot was very tight.

The project vision emerged from the following business challenges:

• Needing a better digital engagement and digital experience to empower tournament visitors with timely answers to their questions based on the most recent information.
• Requiring more efficient usage of data collected from digital interactions – with actionable insights that lead to long-term customer retention and upsells.

Project Overview

Neurons Lab created a chatbot that uses GenAI powered by AWS Bedrock in just six weeks. Key features include:

• Conversational AI: The chatbot provided a natural language interface for users to ask questions in different languages and receive informative responses about the tennis tournament.
• Context awareness: By leveraging a stored chat history, the chatbot can maintain conversation context and provide relevant responses based on previous interactions.
• Vector database: Using a vector database and embeddings enables efficient similarity searches, allowing the chatbot to retrieve relevant information from the ingested data.
• Scalability: The architecture scales horizontally by leveraging serverless compute services – AWS Fargate – and load balancing, ensuring the system can handle increasing user traffic.
• Data integration: The solution incorporates various data sources, including web crawling, document storage and other enterprise knowledge bases, ensuring the chatbot has access to comprehensive information about the tennis tournament.

Solution

Solution overview

The system follows a modular approach, with each component serving a specific purpose:

• Data ingestion: The system ingests data from various sources, including web crawling, PDF parsing, and scraping tennis results. This data is then preprocessed and converted into LangChain documents, a framework for building applications with large language models (LLMs).
• Vector store: The preprocessed documents are embedded using AWS Bedrock Embeddings, and the resulting vector representations are stored in a PostgreSQL vector store.
• Retrieval Augmented Generation (RAG): The core AI component uses a RAG approach that combines a vector database lookup with a language model to generate responses.
• LangChain components: The system utilizes various LangChain components, including prompts, chains, runnables, and callbacks, to orchestrate the AI pipeline.
• LangServe API: The AI chain is exposed as an API using the LangServe library, allowing users to interact with the chatbot.

Data flow

A user seeking information about the tennis tournament sends a query using text or audio via WhatsApp. Then:

1. This query is routed to the WhatsApp application via the Load Balancer.
2. WhatsApp interacts with the API application which handles the core chatbot functionality.
3. The API application leverages Amazon Bedrock, utilizing Anthropic AI and natural AI models to generate responses and process user inputs.
4. Relevant data and context are retrieved from the Vector database – Aurora – using vector similarity searches facilitated by the Amazon Bedrock Embeddings service.
5. Redis manages and stores chat histories, enabling seamless conversation tracking and context preservation.
6. The API Application integrates with various data sources, such as Google Drive, the web crawler, and Amazon S3, to ingest and process relevant information about the tennis tournament.
7. Responses generated by the LLMs are sent back to the user through the WhatsApp interface.

Evaluation

The codebase includes an evaluation component that leverages the LangSmith and custom-developed evaluation metrics to assess the chatbot system performance.

These custom evaluators use prompt templates to generate instructions for the LLM to evaluate the chatbot’s performance based on different criteria. Key components include:

• Dataset creation: The code creates a dataset using LangSmith. This dataset contains a list of input questions related to the tennis tournament. Example inputs include: “Where can I buy tickets?” and “How much does [merchandise] cost?”
• Remote chatbot integration: The code integrates with the remote chatbot API using a component from LangServe. It invokes the chatbot API and obtains responses for the dataset inputs while generating a random user ID for each request.
• Evaluation configuration: This configuration evaluates the relevance of the retrieved context to the given query and checks if the provided context supports the claims made in the response. It also considers the relevance of the chatbot’s response to the original query and checks whether it is in the same language.

Results

The visualizations above show several metrics for user engagement with the chatbot – including some of the most common query types and the average message length.

The results dashboard – for stakeholders, analysts, and chatbot administrators – provides insights into user interactions, behavior, sentiment, topics, and engagement. 

Its abilities include but are not limited to:

• Analyzing user engagement
• Analyzing the sentiment of user messages
• Extracting and visualizing the top five tennis player names mentioned
• Separating messages by language
• Extracting top keywords from queries
• Summarizing keywords into topics, as shown above
• Creating word clouds, as shown below
• Pre-processing user messages by removing stop words, punctuation, and caps lock

The visualization above shows one example of a word cloud detailing some of the most common keyword queries the chatbot received during a set timeframe.

In summary:

• The final product, delivered in record time, was a GenAI-based chatbot that uses structured data to answer questions and maintain conversation via WhatsApp.
• It organized knowledge based on several forms of data, including the tennis tournament website and closed information sources, as a vector database ready for semantic search and retrieval.
• It provided an improved user experience and digital engagement for the tournament and empowered visitors with timely answers to their questions based on live information

• The chatbot also built customer profiles to support upsell opportunities and customer retention for future tournaments.

About Neurons Lab

Neurons Lab is an AI consultancy that provides end-to-end services – from identifying high-impact AI applications to integrating and scaling the technology. We empower companies to capitalize on AI’s capabilities.

As an AWS Advanced Partner, our global team comprises data scientists, subject matter experts, and cloud specialists supported by an extensive talent pool of 500 experts. We solve the most complex AI challenges, mobilizing and delivering with outstanding speed to support urgent priorities and strategic long-term needs.

Ready to leverage AI for your business? Get in touch with the team here.