Regarding software development, it’s important to remember that most applications are written in multiple languages.
So anyone on the internet screaming, “ChatGPT was written only in Python!” is entirely wrong.
ChatGPT, being more than just a model, is a very complex software application with many moving parts.
Although the entire application may not have been written in Python, we had to do some digging and found that certain components were.
To determine whether or not this was the case, we conducted thorough research and found that one of ChatGPT’s close companions had actually written their API entirely in Python – and released the source code on GitHub.
As such, we can confidently say that the model building and model training part of ChatGPT is written using the Python programming language.
Below, we go over how we figured this out and reveal the close companion we used to reverse engineer this discovery.
This one gets interesting…
How Software Applications are Generally Created
Software applications are generally created by writing code. (shocker)
However, the type of code one writes is usually broken into two categories: front end and back end.
The Front-End
The front end is the user-facing part of the application; this is what you see when you head over to chat.openai.com and start sending your queries.
These interfaces are built by front-end engineers (again, shocker) and designers.
Designers choose how the app should look, and these front-end engineers code to those design specifications.
This type of coding is predominantly JavaScript, HTML, and CSS.
The Back-End
The back end consists of databases, servers, and code that bring everything together.
Most of the code for the back end will be plugging the front end into APIs (Application Programming Interface)
Think about this as the plumbing in your house, where every one of your guests can see your sink (the front end), but the “stuff” that makes it work is the pipes connected to your local water plant.
That local water plant is what we software engineers like to call an API.
**Note1: We’re assuming this is a microservice architecture, most modern-day software is built using this paradigm.
**Note2: It’s impossible to say what language back-ends are written in, as it’s a design choice, and there are TONS of languages in which you can build a back-end. However, Python is heavily favored for back-end development.
APIs
Without turning this into a software BootCamp, think of an API as simply “doing something.”
When you turn on your sink (front end), the user (you) basically tells the API to do something (send water). This water travels through the pipes (the back end) to serve you the water at the sink.
APIs can get complicated, as you can train a computer to do many things. (basically anything)
One of those things, for example, can be serving a machine learning model.
Currently, Python dominates the Machine Learning Realm, and academia and private companies build their models using it.
I currently work as a machine learning engineer, and we’d never even think of building and training a model outside of Python.
This ties us into ChatGPT, where we know that the front end was not built using Python; we’re not sure if the back-end was built using Python, but we’re positive that the API serving the model was built using Python.
And Heres Why
How We Found Out ChatGPT Model Was Trained In Python
ChatGPT is a product from OpenAI, and they’re not just going to come out and tell us exactly how they built ChatGPT.
As a company, this wouldn’t make any sense, and if you’re trying to generate money from your product, the last thing you want is people creating competition products.
So, how do we know that ChatGPT, the technology that has taken the world by storm since 2022, was built and trained using Python?
Simply by looking at their close friend.
In December, we know Microsoft released that they were pseudo-acquiring ChatGPT and bringing OpenAIs models into things like Bing and other Microsoft-owned products.
Microsoft engineers and OpenAI engineers have been working around the clock to release ChatGPT into search sometime in 2023.
And while doing this, they’ve created some other products together.
BioGPT, Both The Secret And The Answer
BioGPT was released in late 2022.
While this article isn’t about BioGPT, it’s basically an LLM (Large Langauge Model) (like ChatGPT) trained specifically on biomedical data.
What’s even more interesting is that this model is open-sourced, meaning you can download it right now and start working with it.
And what’s even better than that is that they’ve released all of the code from BioGPT.
Since we know that OpenAI and Microsoft are working together on building ChatGPT, we can assume that they would make BioGPT (very similar to ChatGPT) the exact same way.
So, then how do we know that BioGPT was written in Python?
By simply looking.
(Check out the bottom right of this image)
The official Microsoft Repo shows that BioGPT (Remember, the API/Model) is 98.8% written in Python.
Judging by this and knowing how close the relationship is between OpenAI and Microsoft, we know that the model behind ChatGPT is around 99% Python Code.
This shouldn’t shock you if you’ve done any machine learning before.
Python dominates the Machine Learning world, and if you’re a company interested in Data Science, artificial intelligence, and machine learning, you’re probably leveraging Python.
What Else Does This Tell us?
BioGPT tells us a ton more about ChatGPT if you’re willing to look.
We can assume that ChatGPT is a Pytorch Model (Compared to a TensorFlow Model) and that these LLMs hover around above 81% accuracy.
BioGPT only has 1.5B parameters, compared to ChatGPT, which is rumored to have over 10x this amount.
If BioGPT is scoring an 81% accuracy, what do you think ChatGPT is scoring?
Let us know in the comments!
Reference:
https://github.com/microsoft/BioGPT
https://paperswithcode.com/paper/biogpt-generative-pre-trained-transformer-for
Other ChatGPT Articles
We know that ChatGPT has taken the world by storm. And by doing this, it’s also created a TON of confusion.
Don’t worry; we have some guides to help you.
