Conversational/Generative AI Specialist

Core Parts of the Role would be:

• Conversational AI Expertise:
• Understand the latest topics and trends in the conversational AI landscape to identify opportunities to leverage these advancements to gain a competitive advantage, drive efficiency or innovation
• Determine how to best leverage conversational AI to improve agent and customer experience – collaborate with the Product Owner on the AI roadmap
• Build use cases for a chatbot in extended capacities (e.g. authenticated, transaction, financial advice, agent-focused)
• Responsible for testing and certifying certain chatbot features for production deployment, assist in UAT testing and defect management/resolution


• Chatbot Tuning, Continuous Improvements and Enhancement:
• Own model training and tuning through deriving insights from unstructured textual data sources
• Improve accuracy in evaluations; closely monitor and minimize hallucinations and biases
• Work with vendors to understand and tune the underlying large language models and their outputs
• Deploy and test new capabilities
• Identifying opportunities to increase containment rate
• Determine what new data sources can be used to further educate/train the chatbot
• Responsible for the continuous improvements of the machine learning models
• Leverage analytics to get insights on user experience and the performance of the chatbot for continuous improvement
• Monitor and analyze chatbot performance metrics, including response accuracy, customer satisfaction, and escalation rates
• Collect and analyze user feedback and interactions to gain insights into common customer queries, pain points, and emerging trends
• Provide regular updates to stakeholders regarding chatbot performance, enhancements, and any identified issues or challenges
• Develop and maintain comprehensive documentation on the chatbot's functionalities, guidelines, and limitations
• Provide ongoing support and training to agents, addressing queries and issues related to the chatbot


• Risk Management:
• Identify security, regulatory and data privacy issues and develop robust risk mitigation
• Ensure responsible and accountable use of generative AI, addressing issues such as bias, privacy, security and unintended consequences
• Ensure that generative AI applications adhere to robust privacy and security standards
• Ensure responsible AI practices are integrated into implementation plans
• Stay up to date on any impactful legislation (e.g. AIDA Bill C-27, EDGE OSFI)
• Provide effective challenge and liaise identified AI risks to the model owners and developers and conduct analysis of the remediation plans

Lets Talk About You!

• 5-7 years experience
• Bachelor's or Master’s degree in computer science, engineering, mathematics, data science, AI or a related field preferable
• Demonstrable knowledge of and experience with tuning natural language processing (NLP) techniques and tools, which can be used to analyze and understand natural language data; large language models (LLM) and prompt engineering
• Experience training and scaling models, including predictive and prescriptive analytics applications
• Advanced technical skillset required; experience in optimizing, fine tuning and implementing models on information extraction tasks like NER, Summarization, and Question Answering
• Experience in designing and implementing large information retrieval systems using embeddings, vector stores, similarity match, & ranking techniques
• Passionate about generative AI, and has a strong interest in the AI landscape; knowledgeable of trends
• Complex project planning/management skills including the ability to manage deadlines, collaborate with team members, and prioritize tasks
• Excellent problem-solving skills and can think creatively to develop innovative solutions
• Strong communicator who can work collaboratively with cross-functional teams with ease
• Demonstrated ability to own an issue and drive to resolution
• Experience working with external vendors and regulatory bodies
• Excellent verbal and written communication skills in order to help action and facilitate change in the organization towards problem-solving with an AI mindset
• Proven experience working with a high degree of autonomy and self-direction
• Excellent Microsoft Office skills