The Open Source AI Revolution: How History Repeats Itself

Dylan Patel
Semiconductor Analyst
Religion and politics both have a tendency towards polarization, 2023

In the late 20th century, the technology world witnessed a seismic shift as open-source Linux rose to prominence, challenging the dominance of proprietary operating systems from the era’s tech giants. Today, we are on the cusp of a similar revolution in the realm of AI, as open-source language models gain ground on their closed-source counterparts, such as those developed by Google and OpenAI.

In the 1990s, the UNIX ecosystem was dominated by proprietary solutions from major players like Sun Microsystems, IBM, and HP. These companies had developed sophisticated, high-performance systems tailored to the needs of their customers, and they maintained tight control over the source code and licensing. However, Linux, an open-source operating system created by Linus Torvalds, started gaining traction, ultimately disrupting the market.

The Linux revolution was propelled by three key factors: rapid community-driven innovation, cost-effectiveness, and adaptability. By embracing a decentralized development model built off the x86 personal computer, Linux empowered developers worldwide to contribute to its growth. This allowed it to evolve more quickly than its rivals and adapt to a diverse range of applications. Furthermore, Linux’s open-source nature made it significantly more cost-effective than proprietary alternatives, which relied on expensive licensing fees.

Fast-forward to the present day, and we are witnessing a similar upheaval in the AI landscape. The past two months have seen open-source AI projects such as EleutherAI GPT, Stanford Alpaca, Berkeley Koala, and Vicuna GPT, make rapid strides, closing the gap with closed-source solutions from giants like Google and OpenAI. Open-source AI models have become more customizable, more private, and pound-for-pound more capable than their proprietary counterparts. Their adoption has been fueled by the advent of powerful foundation models like Meta’s LLaMA, which was leaked to the public and triggered a wave of innovation. 

The Linux saga offers important lessons for the AI community, as the similarities between the rise of Linux and the current open-source AI renaissance are striking. Just as Linux thrived on rapid community-driven innovation built off the backs of the x86 PC, open-source AI benefits from a global pool of developers and researchers who build upon each other’s work in a collaborative manner off the backs of gaming GPUs. This results in a breadth-first exploration of the solution space that far outpaces the capabilities of closed-source organizations.

Another parallel is the cost-effectiveness of open-source AI. Techniques such as low-rank adaptation (LoRA) have made it possible to fine-tune models at a fraction of the cost and time previously required. This has lowered the barrier to entry for AI experimentation, allowing individuals with powerful laptops to participate, driving further innovation.

Moreover, open-source AI models are highly adaptable. The same factors that make them cost-effective also make them easy to iterate upon and customize for specific use cases. This flexibility enables open-source AI to cater to niche markets and stay abreast of the latest developments in the field, much like Linux did with diverse applications.

Lorem FPO

The implications of this open-source AI revolution are profound, especially for closed-source organizations like Google and OpenAI. As the quality gap between proprietary and open-source models continues to shrink, customers will increasingly opt for free, unrestricted alternatives. The experience of proprietary UNIX-based systems in the face of Linux’s rise serves as a stark reminder of the perils of ignoring this trend. In fact, with image generation bots, OpenAI’s Dall-E and Google’s various closed models are barely a point of discussion as the world flocked to open Stable Diffusion models.

To avoid being left behind, closed-source AI organizations must adapt their strategies. Embracing the open-source ecosystem, collaborating with the community, and facilitating third-party integrations are crucial steps. By doing so, these organizations can position themselves as leaders in the AI space, shaping the narrative on cutting-edge ideas and technologies. Companies like Replit, MosaicML, Together.xyz, and Cerebras are doing just that, releasing open-source models, but offering services, finetuning, or operations as a service instead.

The implications of this open-source AI revolution are profound, especially for closed-source organizations like Google and OpenAI. As the quality gap between proprietary and open-source models continues to shrink, customers will increasingly opt for free, unrestricted alternatives.

The flip side of the argument is that this is only possible for a certain model size. There are many emergent behaviors that have only been witnessed on the largest models. While open-source AIs that are an order of magnitude smaller than GPT-3 have already surpassed GPT-3’s quality, this does not necessarily apply to models of the scale of GPT-4 and beyond. With continued scaling in sequence length, parameter count, and training data set sizes, it is possible the gap between open-source and closed-source widens again.

Furthermore, while models are free to use, services that are built on top will still require significant investments. Google, Microsoft, and Meta are able to build these closed-source services for use in people’s everyday lives due to the moat of their platforms. Lastly, the cost of inference is a significant barrier given most consumer devices do not have the horsepower required for models larger than 7 billion parameters (GPT-3 is 175 billion parameters, GPT-4 is over 1 trillion), and it is possible that only the largest organizations can afford to scale their model out to billions of users.

COUNTER/PARTIES

Against AI
Artificial “Art

Molly Crabapple
Artist and writer in New York City. 

Sure, generative AI threatens mass unemployment for millions of people, far beyond the illustration field, but appeals to ethics and solidarity won’t stop you from using it any more than you from using stopped Amazon or Uber or AirBnB.

You need stronger stuff.

or

The Shift from Knowledge to Direction Work

Toby Daniels
Toby Daniels
ON_Discourse Co-Founder

Shifts in many job sectors do not just show AI replacing human skills. They show a need for a new kind of human skill set. This is where the direction worker comes in.

The SEC’s DeFi moves could push crypto innovation abroad

Ernie Smith
Ernie Smith is the founder
of the newsletter Tedium
and a regular contributor
to Vice’s Motherboard.

Efforts by the Securities and Exchange Commission to take a more direct approach to regulating the emerging field of decentralized finance lack teeth when one of DeFi’s big benefits is mobility across borders.

Does DeFi lose some of its luster if regulators perceive it as a securities exchange? Or is it the U.S. that loses its luster in the increasingly global view of crypto?

That concern has been raised in the wake of recent commentary coming from the Securities and Exchange Commission, which has been looking closely at possible avenues of regulation for the crypto space, particularly around how an “exchange” is defined in the eyes of the law.

The VC giant Andreessen Horowitz had asked, in the wake of the SEC’s rule-making last year, to make it clear that the rule changes did not apply to DeFi.

“We believe that, if adopted, the unnecessarily broad language contained in the Proposal could be interpreted as applying to a broad array of technologies, including DeFi systems and protocols,” the company wrote.

The SEC, in a supplemental document submitted last month, emphasized that it said what it meant when it announced its plan to expand the definition of an exchange.

The commission emphasized that the rules would apply “if these electronic messages constitute a firm willingness to buy or sell a security, including a crypto asset security.”

There’s a strong case to strengthen regulations if the concern is consumer safety. The demise of the FTX exchange, and the reputational hit that comes with that, has put the crypto space back on its heels when it comes to pushback on stronger protections.

“As evident by numerous crypto scandals in the last year—most notably the collapse of the FTX crypto exchange—the industry hasn’t been able to lean into DeFi’s inherent advantages.”
– Umee’s Brent Xu wrote for Coindesk back in February.

Some observers, like Bloomberg’s Andy Mukherjee, saw opportunity in FTX’s demise to remove centralizing elements that are in opposition to the core concept behind decentralized finance.

If the SEC decides to push towards increased regulation, that innovation could happen elsewhere.

Could Exchanges Just … Leave?

The SEC’s approach to DeFi has been seen by some observers as old-school, and have little meaning in a world where borders matter less than ever.

If regulations put exchanges in regulatory jeopardy, it’s quite possible that they could depart to countries that are friendlier to their business models. Hugo Volz Oliveira, the secretary of the Portugal-based New Economy Institute, argued that while the SEC’s actions represent “a clear regulatory attack on the industry,” they ultimately will struggle to rein in the innovation happening in the field. “If any of these vectors is successful, it will certainly lead DeFi platforms and ecosystems to move abroad unless further regulation clarifies the overall situation. While this is particularly problematic for US organizations and individuals, it won’t affect crypto innovation around the world given that this workforce is highly mobile.” – Hugo Volz Oliveira, the secretary of the Portugal-based New Economy Institute

That level of mobility is one of the big advantages of decentralized finance, and there’s a strong case that this could legitimately happen.

In recent months, more centralized U.S.-based firms like Ripple and Coinbase started making waves about potentially lessening their presence in the United States in an effort to get around unfavorable an unfavorable regulatory environment.

That wouldn’t leave the SEC without regulatory options if a rogue player emerges, for example—it could work with outside governments to help rein in potential issues of fraud, or take steps to curb illegal activity that is affecting Americans. (After all, FTX was based in The Bahamas, and the SEC still stepped in.)

It, nonetheless, could discourage some major players in the crypto space from staying in the U.S. market.

“I think if a number of years go by where we don’t see regulatory clarity emerge in the US, we may have to consider investing more in other regions of the world,”
– Coinbase CEO Brian Armstrong recently said at a conference in London, according to Investopedia.

The SEC’s mindset reflects a desire to protect consumers—understandable, given headlines that constantly highlight the risks of investing in crypto.

It also highlights the tough stance on DeFi taken by SEC Chair Gary Gensler, exemplified by this recent comment: “These platforms are acting as if they have a choice to comply with our laws. They don’t.”

It’s a stance that could lead some DeFi players to sue the SEC over the perceived arbitrary approach to defining the terminology around exchanges.

But even if legal action doesn’t go the sector’s way, there is always plenty of room elsewhere. Volz Oliveira noted that the United Kingdom and European Union have each taken steps to better understand the needs of the emerging technology, and would be most likely to benefit if the U.S. hardened its regulatory stance on decentralized finance. “The US surely stands to lose its role as a key builder of future digital technologies and economies if it stays within this course of action” – Hugo Volz Oliveira

This post is for members only.

To keep reading this post, apply to join our Member Waitlist. You may view our membership packages to know more.

Already a member? Login

This post is for members only.

To keep reading this post, apply to join our Member Waitlist. You may view our membership packages to know more.

Already a member? Login