Open-Source Design Platform of 600k Users Introduces GenAI Co-Pilot & Documentation Engine

Penpot partnered with Neurons Lab to transform creative capabilities for their open-source design platform, leveraging generative AI technologies.

Partner Overview

With a growing user base of 600k+ Penpot is the first open-source collaborative design and code tool.

Their mission is to provide an open-source and open-standards platform to take collaboration between designers and developers to a new level. Their aim is to be the ultimate ally for designers, enabling them to achieve unprecedented scalability and creativity.

Having scored $20m in funding last year, Penpot partnered with Neurons Lab to develop new design possibilities for their users through generative AI technologies. This ongoing project began with a deep-dive R&D approach and has evolved into the integration of Penpot’s first AI-based design feature.

Project Objectives 

Penpot had some key objectives in mind for their AI project:

• Firstly, they wanted to empower designers to create in a more efficient way so they could dedicate more time to creative tasks. 
• Secondly, being an open-source platform, they wanted to strengthen and bring more value to their community by building community-powered and owned design systems. 
• Thirdly, they wanted to integrate AI into the product in order to stay competitive. Some of the features they looked to develop were first of their kind and would give them a serious competitive edge if successful. 

And when Penpot got in touch with Neurons Lab, they already had a series of AI features in mind that they wanted to integrate into their design platform.

After assessing feasibility and potential impact on users and business, two features were selected to work on first:

1. AI-Powered Design Co-Pilot

Penpot wanted to build an innovative Design Co-Pilot powered by AI which would allow users to describe desired changes in a text format, leaving the heavy lifting of SVG element editing to the AI system. The Co-Pilot would streamline the editing process, eliminating the need for manual adjustments.

2. UX2Doc feature 

Penpot also wanted to create a UX2Doc feature to transform design ideas into comprehensive documents using generative AI. This feature would automatically generate technical and functional UI/UX specifications, freeing up time for designers to focus on creative tasks.

Business Challenges

Although Penpot had already identified the AI features they wanted to work on, they faced a few challenges along the way. 

Firstly, Penpot could not just buy an “off-the-shelf” solution because they wanted to create an open-source solution that would be available to their community. Furthermore, the necessary open-source datasets don’t currently exist online, which means there was a limited dataset for training, the data would need to be created synthetically. 

In addition, at the start of the project, Penpot’s tech team did not possess the necessary expertise to develop the kind of features they aspired to build. 

Solution Overview

During the ongoing partnership between Penpot and Neurons Lab, fundamental progress has been made in order to research and integrate the two priority AI-based features: AI-Powered Design Co-Pilot and UX2Doc feature. 

Before beginning development on specific features, Neurons Lab created an MLOps pipeline and internal toolset for all of Penpot’s future AI development. This provided an experimentation framework where Penpot could quickly run different experiments and develop new AI solutions.

AI-Powered Design Co-Pilot

Since there is no existing technology that could meet Penpot’s requirements for the design co-pilot, the solution needed to be developed from the ground up. 

After exploring several options, Neurons Lab chose to leverage state-of-the-art techniques by integrating Diffusion-based Models and Large Language Models (LLM) into the AI Co-Pilot solution.

From refining icons to handling complex SVG manipulations, the combination of creativity and technology empowers users to work faster and optimize design outputs. 

This feature is now an advanced prototype, having created a data feedback loop with customers to refine the model. 

UX2Doc feature

Now at the commercialization phase, Neurons Lab created the UX2Doc feature to transform design ideas into comprehensive documents using generative AI. The feature uses Amazon SageMaker JumpStart and Llama 2 to automatically generate technical and functional UI/UX specifications. 

With the feature, developers can do documentation in real-time and focus more on creative tasks, vastly improving the productivity of both individuals and entire teams. The feature has already been deployed to end users.

Results 

• Penpot is now able to quickly run different experiments and develop new AI solutions thanks to their new MLOps pipeline
• Penpot has an advanced prototype for its Co-Pilot solution
• Penpot users can now access a fully functioning UX solution which increases the quality and efficiency of documentation

Next Steps

• Penpot continues to work on its AI efforts, planning the development of their next features
• Neurons Lab will build a Data Science Center of Excellence for Penpot so they can continue AI solution development in-house
• Neurons Lab are currently gathering datasets, training models, and continuing development on Penpot’s Co-Pilot solution

I love how Neurons Lab is able to combine a very results-oriented methodology with the built-in uncertainty of pure AI research. They are able to achieve this by making sure they have picked your brain as much as possible. They have been key in assisting us in some of our strategic decision-making process and they have become our go-to critical friend in ML/AI.

Pablo Ruiz-Múzquiz CEO | Penpot (Kaleidos/Taiga)

About us: Neurons Lab

Neurons Lab delivers AI transformation services to guide enterprises into the new era of AI. Our approach covers the complete AI spectrum, combining leadership alignment with technology integration to deliver measurable outcomes.

As an AWS Advanced Partner and GenAI competency holder, we have successfully delivered tailored AI solutions to over 100 clients, including Fortune 500 companies and governmental organizations.