Team 1 – J.B. Hunt Carrier 360 Mobile Virtual Assistant

CSCE Capstone

Student Site for Individual and Collaborative Activites

Team 1 – Carrier 360 Mobile Virtual Assistant

Team Members:

Description:

The J.B. Hunt Carrier 360 Mobile Application allows drivers to search, bid, and book loads. This application allows drivers to easily find the jobs they want at the prices they want, which greatly improves their efficiency and reduces their down time. However, due to safety concerns, the application locks down when the drivers are on the road. This prevents them from operating their devices while driving, but does necessitate that they pullover and stop while trying to find and bid on their next job. We have implemented a voice interface for the J.B. Hunt Carrier 360 Mobile Application, so that drivers will be able to leverage the functionality of the application, without having to pull over and wait until they successfully get a load. We built the application using a client-server architecture, which was done to conserve client resources and to implement a solution using Microsoft’s machine learning natural language understanding service: LUIS. Our client was built using the React-Native framework and the back-end was implemented with the Azure Bot framework. Due to security concerns with using J.B. Hunt’s live API endpoints, we were given mock data to test and implement our application. With this voice interface, we hope to alleviate pain points of J.B. Hunt’s current users and further improve the functionality and usefulness of the J.B. Hunt Carrier 360 Mobile Application.

Task Schedule:

Tasks	Assigned to	Dates
1. Gain background knowledge in frameworks and SDKs	Everyone	8/26 – 9/16
2. Design client app	Alicia, Anjan, Matthew	9/17 – 10/1
3. Implement client app	Alicia, Anjan, Matthew	10/2 – 10/30
4. Design bot server	Caleb, Brandon	9/17 – 10/30
5. Design and train LUIS agent	Caleb, Brandon	9/17 – 10/30
6. Integrate analytics	Everyone	10/31 – 11/14
7. Deploy, test, and document	Everyone	11/15 – 12/13

The following tasks were completed:

Gain background knowledge of the required frameworks and APIs for the project and setup project environment
1. Understand React-Native framework and the following libraries/middleware
  1. React-Redux for application state management
  2. Redux-Thunk (potentially Redux-Saga) for async action handling
  3. react-native-voice for converting user speech to text
  4. react-native-tts for converting text to speech for relaying messages from the server to the user
  5. ESLint for code styling rules
  6. NativeBase for basic, customizable components
  7. node-fetch for http requests
2. Understand LUIS to build/train a language agent to extract useful text entities from identified intents produced from user utterances
3. Understand Azure Bot Framework SDK to design conversation flow modules
4. Understand Firebase Analytics SDK to integrate analytics into project
5. Create a project repository using Azure DevOps and establish code rules and PR policies
Design client app
1. Initialize a React Native project and install necessary libraries, establish organizational and code styling guidelines via ESLint and a clear readme for easier collaboration
2. Design app interface by developing mocks and use case scenarios
3. Determine the state tree for the app
Implement client app design
1. Develop screen components and style using StyleSheet attributes and components from NativeBase library
2. Integrate react-native-voice and react-native-tts into project to capture voice data, convert to text, then back to voice data.
3. Integrate React-Redux into app to manage state
  1. Develop synchronous and asynchronous action creators to dispatch actions
  2. Develop reducers to capture actions and update app state
4. Use DirectLine REST API and node-fetch to send user utterances to the Node.js bot server, and receive text responses from the server
Design bot server
1. Initialize a Node.js restify server and integrate Bot Framework Service middleware
2. Implement prompts and dialog workflows using Bot Framework SDK dialog library
3. Connect LUIS to the bot service to extract user intents and entities for usage in the dialog workflows
4. Implement search functionality that will import provided fixtures to search against the user queries
5. Capture response data from fulfillment and package up into a text response to be sent back to the client
Design and train LUIS voice agent
1. Create LUIS app and add app credentials to bot server
2. Create intents to identify possible user utterances
  1. Develop fallback intents for unknown or irrelevant utterances
3. Apply intent entities to identify key information within utterances
Deployment, testing, and documentation
1. Deploy bot server to azure
2. Test and debug bot server individually using Bot Framework Emulator
3. Test and debug client app individually using react-native-debugger
4. Test all possible dialogue workflows with client app and bot server integrated.
  1. Test “happy-path” scenarios, where dialog workflow criteria is successfully met
  2. Test “sad-path” scenarios, where dialog workflow criteria is not met at various stages, and fallback scenarios successfully handle these situations
5. Test analytics is detecting app usage and displays meaningful telemetry
6. Generate documentation for final project

Project Proposal Material:

Project Proposal Paper

Project Proposal Slides