Facebook has made its artificial intelligence platform Horizon AI, responsible for maximising the power of the social media giant's data, completely open-source and free to download for anyone in the world

what is horizon ai - reinforcement learning

Apparently in a bid to arm people and companies around the world with a way to make the most of of their growing data sets, Facebook is opening up its artificial intelligence platform to the public – but what is Horizon AI?

The end-to-end platform was designed by the social media giant to train its own computer systems to analyse data and use the feedback to maximise rewards from its own decision-making process.

This process is called reinforcement learning (RL), and it rids an organisation of the need to create tailored policies for using data with continual human input by creating a process that can adapt and make its own decisions.

In an announcement on 1 November, Facebook’s website read: “Today we are open-sourcing Horizon, an end-to-end platform that uses RL to optimise products and services used by billions of people.

“We developed this platform to bridge the gap between RL’s growing impact in research and its traditionally narrow range of uses in production.

“We deployed Horizon at Facebook over the past year, improving the platform’s ability to adapt RL’s decision-based approach to large-scale applications.

“While others have worked on applications for RL, Horizon is the first open source RL platform for production.”

Mark Zuckerberg, Facebook
Facebook co-founder Mark Zuckerberg poses a question during the 2015 CEO Summit. (Official White House Photo by Pete Souza)

What is Horizon AI? What is reinforcement learning?

The term AI is an umbrella for a number of different computer learning techniques, including machine learning and deep learning – and reinforcement learning is now earning its own status as a buzzword.

But it actually pre-dates the hyper-digital age we live in now.

Google research scientist Kevin Murphy, writing in 1998 when he was associate professor of computer science and statistics at the University of British Columbia, explained RL as the problem of getting an agent to act in the world so as to maximise its rewards.

“For example, consider teaching a dog a new trick: you cannot tell it what to do, but you can reward or punish it if it does the right or wrong thing,” he said.

“It has to figure out what it did that made it get the reward or punishment, which is known as the credit assignment problem.

“We can use a similar method to train computers to do many tasks, such as playing backgammon or chess, scheduling jobs, and controlling robot limbs.”


What is Horizon AI? Why Facebook open-sourced it

Other RL platforms exist, but Facebook‘s version is unique is so far as it is designed with the intention to provide production outputs and results – rather than being used for experimentation.

It informs the system behind the Facebook push notification system, where a page administrator sends its users updates relevant to them based on their data, and has been used to improve the quality of key website features such as online video streaming.what is horizon ai

In an interview with venture beat, Facebook engineer and Horizon AI project lead Jason Gauci said: ““I do think RL is kind of the next frontier when it comes to industry-wide, widespread adoption when it comes to machine learning.

“So we wanted to open-source this to really provide a good platform for people all around the [San Francisco] Bay and all around the world to start using it.

“Anyone who has any kind of basic experience can generate a data set and train a model to see how it works, and that’s one of the things.

“There’s sort of an educational aspect to this – we want to get a lot of people kind of excited about the field.”