This document defines the cooperation workflow on AI projects between Streamax and customers. Please read the following content carefully before you work on an AI project or tender as it sets out the terms of necessary actions to ensure the performance.

Before setting off, please fully understand and agree that AI technology product is not the same as other well-defined products, of which features’ performance is with 100% maturity and no need to periodically change. Rather, the biggest advantage of AI technology is with learning ability, the product can be trained by enough footage of actual usage scenarios, to improve the performance better and better.

• Accuracy
  • It’s understandable and reasonable that customers have expectations of AI accuracy rate. We could say that the target can be achieved for DSM and ADAS shortly after training the algorithm, especially for C6D AI, MDVR+AD Kit, AIBOX 4.0.
  • Please note that the algorithm training duration will be longer accordingly for some new AI products.
  • In the beginning, the algorithm may not adapt to local road conditions, in this case, we could use recording metadata provided by our partner to improve the accuracy shortly.
  • If someone irrationally expects the AI system must be 100% accurate, we kindly state that he should not buy AI products. For detailed reasons please refer to the first paragraph.
• Definition of fatigue driving
  • Streamax could detect fatigue driving not only by closed eyes but also by excessive blinking of eyes.
  • Detailed logic: When vehicle speed is above 20km/h, if it’s detected that the driver closes his eyes for 2/2+ seconds, or blinks 10/10+ times within 1 minute and then closes for 0.5s or 1s.

This part is mainly to suggest a better workflow for you, to help with a successful project. Please ensure to align your AI projects info with the Streamax team, especially if you have any new opportunities. Because completed calibration doesn’t mean AI project success, we need to provide support to keep following up with the AI performance and evidence data after kick-off.

Communicate with customers to agree on the expectation, the acceptance should be measurable. Something like, what the customer want is to catch prohibited behaviors as much as possible with tolerable false alarms, or to ensure the accuracy of each alarm with an acceptable false negative. Expectation description should be like this:

• False-positive alarms of any AI alarm type could be less than 30 per day for 1000 vehicles. Based on our experience in past projects, it’s a challenge to make the false rate lower than 3% as the alarming quantity will be sharply decreased after one month of use.
• Get as much project information as possible, including but not limit to vehicle model, target installation location, system architecture, detailed verification schedule, sample vehicle quantity, practical installation, and calibration pictures, etc.
• Communication path
  • A Wechat group should be created for each AI project
  • The support team should include BU, sales team, R&D, especially specific algorithm engineer
  • All of the project info should be passed to the R&D and BU team before POC
• AI system solution, normally our team will help to choose proper solution.
  • Alarm parameters: adjust parameters based on performance expectation, then export the config file and import it to other MDVRs. Back up the config file.
  • Feature verification test must be completed successfully in the office before being put into practical operation. Ensure to have the same model for future testing.
  • Device model, FMW version should be confirmed by BU and R&D team, any change should be confirmed by BU and R&D
• Necessary training:
  • Device installation guide and calibration method
  • How to connect indicator to I/O sensor and related setup
  • Troubleshooting procedure
  • Debugging shouldn’t affect MDVR running
    • FMW upgrade method
    • Speed resource
    • Available config file and setup GUI at any time (key point)
• Checklist:
  • Network status and mobile data
    • Basically speaking, a piece of evidence is about 6M, assuming a MDVR uploads 30 pieces every day and the vehicle runs for 20 days in one month. It turns out that we need 3.6G data for evidence upload, plus basic data upload and recording playback, nearly 8G data is needed (normally it’s two times more than evidence upload’s).
    • What’s the data clearing rule
    • What’s the SIM type? 3G/4G/5G? Is it in VPN?
  • Storage replacement frequency. Please note storage replacement may lead to part of evidence recordings can’t be uploaded to server in time.
  • Volume setup. Suggest to close voice announcement at the early stage, re-open it after accuracy rate tends to be stable as it may bother driving and cause objection.
• Reported to CEIBA II and Crocus platform
  • It’s important to count alarm quantities and get original .264 recording of false alarms for algorithm improvemence.
  • Streamax test server will be best choice.
  • It’s okay if you’d like to only use your own server considering data privacy regulation, but please let us know the login account and password.

• Suggest to have a trial running on 1~3 sets, then add sample vehicles gradually according to actual performance
• ST team need to inform the customer ASAP if notices any false alarm due to incorrect camera installation
• To track the accuracy rate of false positive alarms
  • Streamax team will adjust the settings to get all of alarm recording, the evidence data will be uploaded to CEIBA II server. Below is a suggestion about AI evidence uploading.
  • If any setup change or FMW upgrade OTA, Streamax need to inform the customer via e-mail to avoid any misunderstanding on system stability.
• To track the accuracy rate of false negative alarms
  • In POC process, please connect a press button to one of I/O sensor, if the driver notices any false negative, please press the button to trigger a period of I/O alarm recording.
  • Please install two more cameras in the whole system, one is facing R-watch, the other one should face the driver to see his behaviour.
  • Streamax support team will compare the driver’s actual behaviour and the I/O alarm recording to confirm if it’s a real false negative, then take actions.
• Alarm statistics procedure
  • Login CEIBA II server and click to enter alarm centre
  • In ‘Search’ page, tick the target vehicle and choose time range, all alarms will be shown after click ‘search’ button
  • Export alarm list, and re-organise the info into statistic template as below "Alarm report 2020.12.08.xlsx"
  • Check if the quantity of evidences is the same as that of alarm. If not, please check evidence upload strategy then see if some recordings were overwritten before complete uploading. The comparison helps to find out the device abnormality
  • Send the statistic excel and collected false alarm H.264 recordings back to R&D, the accuracy rate is subject to R&D’s statistic result. Any valuable recordings will be used for algorithm training
• Write down any issue found in the POC process to avoid in the future.
• We could say that, AI project can be considered start successfully only when AI performance is good enough after a period of continuous tracking.

Please note that OTA upgrade is necessary for most of AI technology products, so do our competitors. Similarly, Tesla often upgrades their algorithm as well. So we do suggest that every three months or half a year, it would be better to upgrade the algorithm or firmware to get better performance.