The CodeFlare framework can be deployed almost anywhere to deliver serverless benefits

1200px-2017_sediul_IBM

IBM launches a new open-source framework called CodeFlare. (Credit: Babu/Wikimedia Commons)

IBM has launched an open-source framework called CodeFlare for simplifying the integration, scaling, and speeding up of complex multi-step analytics and machine learning pipelines on the hybrid multi-cloud.

CodeFlare has been built on top of Ray on IBM Cloud Code Engine, which is an open-source distributed computing framework for machine learning applications.

The new open-source framework is said to extend Ray’s capabilities through the addition of specific elements for making scaling workflows easier.

According to IBM, for creating a machine learning model, researchers and developers are currently required to first train and optimise the model. The process is said to be simplified by CodeFlare by using a Python-based interface.

The new framework’s objective is for unifying pipeline workflows across various platforms without the need for data scientists learning a new workflow language.

IBM stated: “Data and machine-learning analytics are proliferating into just about every industry, with tasks becoming ever-more complex. Larger datasets and more systems designed for AI research are fantastic—but as these workflows become more involved, researchers are spending more and more time configuring their setups than getting data science done.”

As per the company’s research, when a user applied its new framework to analyse and optimise nearly 100,000 pipelines for training machine learning models, the time taken for executing each of the pipelines was reduced from four hours to 15 minutes.

IBM said that CodeFlare pipelines run with ease on IBM Cloud Code Engine, which is the company’s new serverless platform, and on Red Hat OpenShift as well. This means users can deploy the framework almost anywhere, thereby extending the advantages of serverless to data scientists and researchers of artificial intelligence (AI).

The company stated: “It also makes it easier to integrate and bridge with other cloud-native ecosystems by providing adapters to event-triggers (such as the arrival of a new file), and load and partition data from a wide range of sources, such as cloud object storages, data lakes, and distributed filesystems.”