How to Know When Products Actually Use AI

by Ivan Novikov3/10/2017

Artificial Intelligence has become a buzzword. People use it in sales pitches all the time and you often see it in ad copy for new gadgets and apps. It also happens to be the most important tool in the cybersecurity field.

Here you will learn how to distinguish between a service provider that is truly using artificial intelligence in their product – and has enough big data that it helps them form such technology – from those that are simply using the allure of the term AI to attract new customers. This could also help investors understand the real state of AI at the vendor/startup side.

Below are five questions to ask when evaluating an AI product.

1. Can the company give you a stand-alone demo?
To avoid someone manipulating your data in the cloud, ask for a stand-alone demo so you can be sure it’s actually software that’s processing your data. If the vendor can only process data in the cloud, you should evaluate the product with a massive amount of data to be sure that it’s impossible to process by analysts.

2. Can you use your own data?
When vendors provide demos using their own data, their system tends to work very well. That doesn’t mean that the same system will work perfectly with your data. My recommendation is to test the system in real time using your data to make sure that it would work in a real life scenario.

3. What are their data sources and sizes?
AI can’t become real AI without big data; this would be like a human surviving without oxygen. For effective work and development, there should be real, big data provided and the vendor should provide the exact numbers, parameters, and available capacity. It’s really important to know and proof data sources.

Common sources: In most cases vendors are using public sources such as stocks history, government data, or open source collections. The brilliant resources here are:

4. What are the details of the algorithm?
Ask the vendor for the details of the implemented approach (algorithm). What exactly (what data) is encoded and decoded. For example, find out how a recurrent neural network is implemented in the product. Approaches aren’t a trade secret so your vendor will be able to discuss them. If the vendor won’t discuss implemented technological approaches, it’s a red flag.

5. Do they have reference customers?
Talk to people already using the product and find out how they’re using it. For example, for security there is a huge difference between a blocking mode and just monitoring. Only a blocking mode could affect the business in case of a false positive event. Try to find customers with needs that are similar to yours.

A company that’s actually using AI and has a high standard of quality should be able to easily answer each of these questions.

We’re still in the early stages of AI development and I can’t wait to see how things unfold.

Good luck!




  • Ivan Novikov

    Ivan is the CEO of Wallarm (YC S16). Wallarm is a DevOps-friendly Web Application Firewall (WAF). Founded in 2013, Wallarm provides a unique hybrid application firewall solution with adoptive nodes i