Thursday, February 29, 2024
HomeLearning TechBeyond Human: Generative AI And ChatGPT Phenomenon Deeply Explained

Beyond Human: Generative AI And ChatGPT Phenomenon Deeply Explained


Have you ever wondered how do powerful generative AI systems such as ChatGPT (a large language model-based chatbot) work, and what makes them different from other types of artificial intelligence?

Understanding Generative AI

But, what does it really mean when we say “generative AI?”

Before the generative AI boom of the past few years, when people talked about AI, typically they were talking about machine-learning models that can learn to make a prediction based on data. Such models were trained, using millions of examples, to predict, for instance, whether a certain X-ray shows signs of a tumor or if a particular borrower is likely to default on a loan.

You can think of generative AI as a machine-learning model that is trained to create new data, rather than making a prediction about a specific dataset. A generative AI system is the one that learns to generate more objects that look like the data it was trained on. The recent buzz around generative AI has been driven by the simplicity of new user interfaces for creating high-quality text, graphics, and videos in a matter of seconds.

Understanding the Historical Context 

It should be noted that the concept of generative AI itself isn’t brand new. Generative AI was actually introduced in the 1960s in chatbots. But with the introduction of generative adversarial networks (GANs – a kind of machine-learning algorithm) in June 2014, generative AI progressed to a level where it could create convincingly authentic images, videos, and audio of real people.

So, with all the buzz, hype, attention, and excitement that came with the launch of ChatGPT on November 30, 2022 and its counter parts, it should be well considered that these powerful machine-learning models draw on research and computational advances that go back to more than 50 years.

Early Generative AI

An early example of generative AI is a much simpler model known as a ‘Markov chain’. The technique is named for Andrey Markov, a Russian mathematician. In 1906 he introduced this statistical method to model the behavior of random processes. In machine-learning, Markov models have long been used for next-word prediction tasks, like the autocomplete function in an email program. 

In text prediction, a Markov model generates the next word in a sentence by looking at the previous word or a few previous words. However, these simple models can only look back that far and they aren’t good at generating plausible text. So, in present-day, the major distinction is in terms of the complexity of objects we can generate and the scale at which we can train these models.

If we look a few years back, researchers tended to focus on finding a machine-learning algorithm that could make the best use of some specific dataset. But that focus has now shifted a bit, and many researchers are now using larger datasets, perhaps with hundreds of millions or even billions of data points, to train models that can achieve impressive results.

Recent Developments and Shift in Focus in AI Research

The base models underlying ChatGPT and similar systems work in much the same way as the ‘Markov model’. But one big difference is that ChatGPT is far larger and more complex, with billions of parameters. And it has been trained on an enormous amount of data (much of the publicly available text on the internet).

In this huge corpus of text, words, and sentences appear in sequences with certain dependencies. This recurrence helps the model understand how to cut text into statistical chunks that have some predictability. It learns the patterns of these blocks of text and uses this knowledge to suggest what might come next.

Advancements in Deep-Learning Architectures

While bigger, complex, and sophisticated datasets are one catalyst that led to the generative AI boom, a variety of major research advances also led to more complex deep-learning architectures.

As discussed above, in 2014 a machine-learning architecture known as a generative adversarial network (GAN) was proposed by researchers at the University of Montreal. GANs use two models that work in tandem: One learns to generate a target output (like an image) and the other learns to discriminate true data from the generator’s output. The generator tries to fool the discriminator, and in the process learns to make more realistic outputs. The image generator StyleGAN, which was introduced by ‘Nvidia’ researchers in December 2018, is based on these types of models.

Diffusion models were introduced a year later by researchers at Stanford University and the University of California at Berkeley. By iteratively refining their output, these models learn to generate new data samples that resemble samples in a training dataset, and have been used to create realistic-looking images. A diffusion model is at the heart of the text-to-image generation system Stable Diffusion.

In 2017, researchers at Google introduced the transformer architecture, which has been used to develop large language models, like those that power ChatGPT. In natural language processing, a transformer encodes each word in a corpus of text as a token and then generates an attention map, which captures each token’s relationships with all other tokens. This attention map helps the transformer understand context when it generates new text.

These are only a few of many approaches that can be used for generative AI.

Applications of Generative AI

What all of these approaches have in common is that they convert inputs into a set of tokens, which are numerical representations of chunks of data. As long as your data can be converted into this standard, token format, then in theory, you could apply these methods to generate new data that look similar.

Since, generative AI allows machines to not just learn from data, but to generate new information that’s similar to the input used to train it, the implications are multi-dimensional because the technology can be used in design, music, art, and other areas.

For instance, a group at ‘The Massachusetts Institute of Technology (MIT)’ is using generative AI to create synthetic image data that could be used to train another intelligent system, such as by teaching a computer vision model how to recognize objects.

Similarly, another group at MIT is using generative AI to design novel protein structures or valid crystal structures that specify new materials. The same way a generative model learns the dependencies of language, if it’s shown crystal structures instead, it can learn the relationships that make structures stable and realizable.

Challenges and Issues 

Generative AI chatbots are now being used even in call centers to field questions from human customers, but this application underscores one potential red flag of implementing these models, the worker displacement.

In addition, generative AI can inherit and proliferate biases that exist in training data, or amplify hate speech, and false statements. The models have the capacity to plagiarize, and can generate content that looks like it was produced by a specific human creator, raising potential copyright issues.

Risks of Generative AI

The risks associated with generative AI are significant and evolving fast. A wide array of threat actors have already used the technology to create “deep fakes” or copies of products, and generate artifacts to support increasingly complex scams.

ChatGPT and other similar tools are trained on large amounts of publicly available data. They aren’t designed to be compliant with General Data Protection Regulation (GDPR) and other copyright laws.

Some oversight risks to check include:

  • Lack of transparency. Generative AI and ChatGPT models are unpredictable, and not even the companies behind them always understand everything about how they work.
  • Accuracy. Generative AI systems sometimes produce inaccurate and fabricated answers.
  • Intellectual property (IP). There are currently no verifiable data governance and protection assurances regarding confidential data and information. Users should assume that any data or queries they enter into the ChatGPT and its competitors will become public information.
  • Cybersecurity and fraud. Malicious actors can potentially use generative AI systems for cyber and fraud attacks, such as those that use deep fakes for social engineering of personnel.

The Future of Generative AI

In the future, generative AI will change the economics in many disciplines. It is safe to anticipate that, in the future, generative AI will become a general-purpose technology with an impact similar to that of the steam engine, electricity, and the internet. The hype will subside as the reality of implementation will set in, but the overall impact of generative AI will continue to grow as people and enterprises will discover more innovative applications for the technology in daily work and life.


Please enter your comment!
Please enter your name here



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments