# GPT-4 + Bing Chat + Q&A – LifeArchitect.ai LIVE: Explorando as possibilidades da inteligência artificial
## Introdução: O que são leads orgânicos?
## A diferença entre leads orgânicos e pagos: Vantagens e desvantagens
## Entendendo o funcionamento dos mecanismos de busca: como otimizar seu site para gerar leads orgânicos
### Palavras-chave e pesquisa por voz: como utilizar a inteligência artificial a seu favor
### Estruturação do conteúdo: como criar um site interativo para melhorar a experiência do usuário
### Backlinks e link building: a importância de uma rede de links orgânicos para seu site
### Monitoramento dos resultados: como identificar e corrigir problemas em seu site
## Gerando conteúdo engajador: como utilizar as redes sociais para atrair leads orgânicos
### Compartilhamento de conteúdo: como alavancar seus resultados e aumentar a interação do usuário
### A importância do storytelling: como contar histórias para se conectar emocionalmente com seu público
### Vídeos e imagens: como usar imagens e vídeos para destacar sua mensagem
## Construindo uma lista de e-mails: como gerar leads orgânicos através de e-mails
### Criação de landing pages eficientes: como criar landing pages otimizadas para converter visitantes em leads
### Utilizando forms e pop-ups: como utilizar essas ferramentas para coletar informações dos seus visitantes
### Personalização do e-mail marketing: como utilizar a segmentação de e-mail marketing para aumentar a taxa de conversão
## Conclusão: Como gerar leads orgânicos de forma eficiente
## Perguntas frequentes
### O que são leads orgânicos?
### Qual a diferença entre leads orgânicos e pagos?
### Como faço para otimizar meu site para gerar leads orgânicos?
### Como utilizar as redes sociais para atrair leads orgânicos?
### Como construir uma lista de e-mails eficiente para gerar leads orgânicos?

br>The Memo:
GPT-4:
Bing Chat:
Google Bard:

====
Links:
Microsoft VP ‘not GPT-3.5’:
LLM abilities emergence:
====

Read more:

Dr Alan D. Thompson is a world expert in artificial intelligence (AI), specialising in the augmentation of human intelligence, and advancing the evolution of ‘integrated AI’. Alan’s applied AI research and visualisations are featured across major international media, including citations in the University of Oxford’s debate on AI Ethics in December 2021.

Music:
Under licence.

Liborio Conti – Looking Forward (The Memo outro)

Este vídeo foi indexado através do Youtube link da fonte
gpt 4 ,

gpt3,dall e 2,midjourney,stable diffusion,gpt4,openai,gpt-3 ,

https://www.youtubepp.com/watch?v=3X0yRWRSpz0 ,

World of AI I’m going to be speaking plain language there’ll be some bubbles that pop up to explain or to put in the technical buzzwords but I’m going to try and leave them out right now people are asking how is AI able to look at a photo

Like this how many people are there in this picture there are at least five people in this picture where is Obama’s foot positioned Obama’s foot is positioned on the right side of the scale they’re also asking how is an AI able to answer a question like this what’s the relationship between

Intelligence and communication intelligence is about understanding the world communication is about being understood by the world wow and what about this why the Noble Path I saw the Noble Path and it was like a river flowing with information but then there were shallow Waters with no flow or Direction

In these Waters I could see people drowning in ignorance and poverty they didn’t know what to do to help themselves survive they needed someone else’s hand reaching out for them pulling them into safety but they didn’t have anyone let’s start at the start if you are a professor at a

University that studied artificial intelligence before 2017 or even 2020 and you haven’t caught up since then your entire knowledge is completely irrelevant see it all started in an AI Lab at Google in 2017 they were researching how to translate language from say English to a language that needs gendered nouns think about French

Or German where every noun every thing is either a he or a she how are we even going to figure that out if we’ve got English which doesn’t even have the he or the she before cat or dog or clock or grass how do we translate that

Here’s a really simple version the cat sat on the mat because it was fat as a human being you can probably translate what the word it means here does it refer to the mat or does it refer to the cat probably refers to the cat in fact there’s a high likelihood

In AR we might call that statistics probability that the word it refers to the noun cat let’s try a little tweak on that the cat sat on the mat because it was flat what does the word it mean in this sentence does it refer to the cat or the

Mat we’ve only changed one letter the cat said on the mat because it was flat but we’ve changed the entire meaning of the sentence the word it here probably refers to the mat Google continued this Research into translation they had some big ideas on how we could use translation in language

While looking forward and backward in the sentence they really accidentally Came Upon This new discovery and this new discovery was that they could guess they could predict what word was going to come next but the computers that they were using allowed them to take a whole piece of text and actually guess where

The very good success rate what was going to come up next so this entire concept is called the Transformer it came into being in 2017 and that’s pretty much all it does it looks forward and backward in a sentence for some AI models it only looks forward and it’s really really interesting it’s

It’s created some big waves in the entire world all right let’s take this Transformer model and go in a bit more detail so Google in 2017 came out with this idea and then AI labs around the world started going crazy with it starting with a lab called open AI in

San Francisco they originally trained a Transformer with a whole lot of information so that they could guess what was going to come next for any question not just for a chat bot not just talking to an AI thing or person but for finishing emails creating music generating art you could use transformer

For any anything where you can guess the next thing in a sequence in 2018 they started by feeding this Transformer model with books lots and lots of books tens of thousands of books from your library fiction books then in 2019 they decided what about if we fed it with very popular internet so

Things that people visit not just the entire internet but the stuff that seems to be upvoted then in 2020 they thought what about if we feed it with as much data as we can and to demonstrate this I’m going to use what we call in Australia jelly in fact if you’re an

Australian you’ll remember this ad If you’re American you’re jello stools are finer than sand and I think they’re going to be really useful for us to demonstrate really big data so one jelly Crystal might be the size of an entire web page or even an entire book and show us what could be possible

Through giving a lot of data to a really big model so let’s use airplane jelly I’ve got some blue crystals here airplane jelly blue I think we’ll use as the internet or the web data so let’s give that one a bit of a label this is going to symbolize the entire internet

Web pages every jelly Crystal here is a web we’re going to actually need six parts of this to demonstrate how much web we put into training a model let’s try a different one what about popular web so for example if you go on Facebook or if you

Go on Twitter or if you go on Tick Tock you’re probably going to be referred to things that are a little bit different to the entire Internet it’s been upvoted it’s been popularized it’s been ranked by human beings let’s use Reddit ratings for that one this is orange Jello let’s

Call this one popular and it’s going to be about two parts versus six parts of full web data the popular web data is going to be fed in to this model as well whole bunch of books the fiction books from your library but also massive academic Publications let’s

Grab every research paper that we can find and put it in this training data as well so we use a purple box of jello for this one this is going to be books and this is going to show us everything from scientific literature to Choose Your Own Adventure to Enid Blyton

To Agatha Christie let’s lump it all in here and the last one despite what your teacher may tell you Wikipedia is actually a very good source of information it’s been edited by thousands and thousands of human beings that know what they’re doing any error is shut down immediately it’s actually

Quite factual you should let’s use a green one here for Wikipedia this is Green Jello crystals and that’ll give us a smaller part we’ll just use one part of Wiki this will show us some of the factual content so that gives us one two three four different types of data

And we’re going to put them in different ratios let’s pour them into the jar now our six parts of web might be 100 million web pages just general websites our popular websites might be 8 million web pages that have been upvoted by humans our books and papers are millions of

Books and millions of University papers and our Wikipedia content is about 6 million web pages here’s our final product six parts web two parts popular web two parts books and at the very top there one part of Wikipedia and different labs are using different weightings for this training

Data but let’s take this as our entire training data for our AI the next step is to feed this into the Google Transformer that we just spoke about the Magic Black Box program that can take words make connections between them by looking forward and backward in the

Words and then come out with what some people are just calling magic or miraculous let’s see how close we can go to explaining that and let’s put the whole thing in a magic Black Box because we don’t actually know and the researchers the scientists don’t know what happens they take the training data

They put it in this magic Black Box and they train it for 288 years worth of computing isn’t that absurd so a model trained in California took 288 years of computation to come out with a result one thing we can do though is run it on more than one computer at once

So for example we could run it on a thousand computers at once and instead of 288 years it might just take three months which is exactly what happened they put it in the black box and they put it in a thousand different computers trained connections between all of this

Data and after three months they had a little piece of magic all right my Prodigy clients will recognize this little model it’s a model of the human brain and it’s not very high quality so we’ll use some video alongside this as well the reason that I

Usually have this one is because we can split it open and talk in more detail about the general parts of the human brain but I’ve got it out today so that we can mention the fact that there are 86 billion neurons inside the brain and back into our jar a neuron might be

Compared with a data point a tiny little piece of data after training not before training the human brain connects each of those neurons with synapses and it might be said that we have 500 trillion synapses in our brain these are connections between neurons and for our purposes

Today let’s grab one more tool to describe what’s going on inside the brain here when we’re talking about neurons and connections between neurons which are synapses this thing’s called a squish you usually give it to your baby it’s a it’s a tensegrity toy some myofascial people use it to explain how

The body is connected we’re not going into that level of detail today but if we use our jar that’s been training in here and we say that it’s trained each data point here could be the little dots on the tensegrity model and the connections between the dots could be

The parameters or the synapses in the human brain their indexes their weights their connections between what has happened during training and what the output is hopefully that helps to demonstrate and give an example of that without going to Too Much technical detail it’s important to note that right

Now after three months training or the equivalent of 288 years of training this black box no longer contains the original training data it only contains the parameters the connections between data points so even though we fed it with books those books do not exist in the model anymore only little Snippets

Of text connected to other Snippets of text sometimes with dozens or hundreds of connections to get to an example here might be this black box has 175 billion parameters or synapses or connections and some large AR models now have up to a trillion and we’re breaking that point one trillion connections if

We wanted to copy the human brain and get the same count of neurons with and synapses we would have to get 86 billion in terms of neuron data count and in terms of synapses we’d have to get to 500 trillion parameters so 500 times larger than what we currently have so

After the three months of training the researchers the scientists couldn’t actually tell you the result they know that there’s a black box they know that you can query this black box with anything if it’s got language you can ask it anything or give it a problem and

It will give a response but other than that they don’t actually know what happened in those three months and they don’t know what’s inside the output besides the fact that it exists in fact a whole bunch of very smart scientists at a very big university called Stanford said we currently lack a clear

Understanding of how they work when they fail and what they’re even capable of due to their emergent properties this is about AI models in plain English they’re saying we don’t know how the AI models work in 2022 because it’s so new and there are researchers trying to figure

Out what is in here what’s going on in here but right now we don’t have an explanation beside the fact that it’s made a lot of connections and it’s able to do good morning wherever you are in the world hi to hacks are in Miami Florida

Uh I’m sure I just had this in my head think about why in Nashville I didn’t get to Nashville but apparently it’s beautiful when some hacks in UK BT Franklin I miss my second home in Phoenix already uh Anthony as well in the UK Wherever You Are

Hope you’re having a cool time hi to Grayson in Waco Texas Justin in Canada we’ve got a few Canadians in here which is kind of cool this might be the most important video we’ve done this year 2023 I know we’re only like not even 60 days into the year but we’ve had

A lot happen in the last uh two months we’ve had some really big and efficient models coming out that are using some maybe some Trade Secrets but we’re seeing smaller models packed with more data and trained for longer I’m going to talk today about Bing chat by Microsoft

As well as gpt4 now if you’re a member of the memo you will have seen a week ago on the 20th of February I sent out an Edition that talked about gpt4 being already in use and GPT 5 already training I only use rigorous incredible sources here so let’s scroll

Down a bit here actually let’s scroll underneath so Harvey is one of the guys that are sponsored by open AI or have an open AI investment of between 5 and 10 million dollars a journalist said that Harvey is a verticalized version of what I understand to be gpt4 so it’s got this

Uh legal bent or it’s got this legal Focus that part is kind of interesting I think there’s three things I want to pick up here the first is that when Microsoft CEO was asked what Bing chat is is it gpt4 the CEO said I’m gonna let Sam open AI CEO talk about

That he didn’t come out and say no he also couldn’t confirm it because it’s not his place but I thought that was a fascinating Insight doesn’t prove anything there’s a Morgan Stanley research note here that I’ve put in that says this is via Morgan Stanley they don’t mess

Around we think that GPT 5 is currently being trained on 25 000 gpus which would be nearly a quarter of a billion dollars of Nvidia hardware and they go into a little bit more detail on that one of the references that is even more recent than that is on LinkedIn you can

Go and find this yourself I may or may not have put it in the description of this video but Geordie robust here is the vice president of search and AI at Microsoft and I will make this even bigger because I think it’s worth reading he says last summer open AI so last

Summer is US time that would be like June July 2022. last summer June July 2022 open AI shared their next Generation GPT model with us and it was Game changing the new model was much more powerful than gbt 3.5 which Powers Jeep chat chippy tea

And so they took it and ran with it seeing this new model inspired us to explore how to integrate the GPT capabilities into the Bing search product um interesting not conclusive and today I’m not going to say that Bing chat is gpt4 but we’ve just seen that it’s beyond GPT 3.5 we’ve

Got this crazy view of all 50 plus models in the GPT 3 family if you look kind of down the bottom right there you’ll see text DaVinci 003 175 B highlighted in yellow that’s probably the most popular via API or via the playground then you’ve got chat GPT

Which I’m sure you’ve played with that is the most popular in the world now because UBS told us they have 100 million monthly unique active users chat GPT also featuring on the cover of Time magazine in a couple of days or maybe it’s today and just underneath those two

Yellow highlights it says Microsoft Bing chat related to web GPT is my guess it’s proprietary web GPT is over there on the left but I am guessing that this must be if not gbt 3.9 and then gbt4 let’s see how we can actually have a play around

With that today you can do everything I’m doing today yourself I don’t have any private or confidential insights I’m not working with any of these labs for this conversation I don’t have any insights about gpt4 besides what has been made public and that is kind of the interesting stuff so how many people

Here have got access to the new Bing chat jump into the uh YouTube chat and let me know Kevin says Microsoft strongly hinted Bing was powered by gpt4 yeah that sounds about right and then BT Franklin saying I really hope binchat isn’t gpt4 because it’s not impressive interesting

I found to get access to Bing I had to try a couple of accounts I’ve used I think it’s my Skype account here to get access but we’re going to have a little bit of a play with it and see where it gets to this is the interface I had to

Download and use Microsoft Edge on Mac OS to use it which was a little bit of a surprise but let’s jump in one of the things I’m going to highlight here is that it’s really difficult to test models without API or playground access because we don’t know what temperature chat GPT is

Using the same for Bing chat so we don’t know whether the results that we’re getting uh testable or not repeatable or not they’re probably not repeatable open AI actually recommends running chat with a temperature of 0.9 which is super super super creative the next the only stop past that is 1.0

So you won’t be able to repeat those results a temperature of zero which is what we normally use to test models especially for papers would mean that that result is repeatable it’s the it’s the first response that you’d get every time whereas what we’re going to do today is a little bit more

Casual less scientific let’s put it that way yeah so it’s got limits for of uh it was five conversation turns before it stops and 50 for the entire day I think it was now it’s six conversation turns before it will ask us to shut up normally I

Would test any search engine with this query maybe it looks unusual to you I used to work with a musician here in Australia called Nathan gaunt so what I’m testing you know DuckDuckGo or a different search engine this is the first thing I put in just to see what it

Gives me is it giving me his albums is it giving me uh his location oh sorry I just searched on the term Nathan gaunt usually I’m putting in here something a little bit more funky wow let’s say the singer-songwriter to actually find a location Google would do it so this is a

Pretty basic question but Bing search goes through what I seem to think is Chain of Thought reasoning and it’ll spell out how it got there let’s see what it gets to here according to Wikipedia Nathan was born in Darwin later settled in Perth okay that’s pretty accurate let’s get a

Little bit more funky here I love asking this question you’ll see it in my chat GPT prompt book which is up to a million users now as our estimates a million views as a million viewers sorry create a table showing a few 21st uh century

Movies set in an office at night this is not something that Google would be able to do because it’s language but of course it’s pretty easy for both chat GPT and Bing chat to do this to step through it and then to play around with it

One of the things I liked about Bing chat is that it’s quite expressive you’ll see it give little emojis while it’s spelling things out as well great all right that’s a list of uh what was that four movies in a nice table does quite well that’s not particularly

Impressive because you can do the same thing in chat GPT I love that and it will give a very similar result so similar that some of those movies look exactly the same Belco and the call I think were in yeah there we go we’re in Bing’s result all right so here’s

Where we get a little bit more clever we’ll actually ask Bing to help us out here when I worked in gifted research in intelligence research we would talk about this concept for very very smart kids for Prodigy level children we’ll talk about this concept of above level testing

And an example of that would be giving an eight-year-old an SAT test the reason we do that is because if we had a one meter rule on a wall that kids in primary school walk up to we would know which children are 90 centimeters 95 centimeters 99

Centimeters and 100 if you put a very tall child against that wall you’re not going to know how tall they are just that you’ll know that they’re more than one meter so above level testing brings in a completely different ruler so you might give them an exam an exam or a

University level test that’s what we’re going to play with here with Bing search we’re actually actually going to bring in the Google Palm paper which I think will be fun first we’ll ask Bing to explain above level testing and we’ll see if it gives a different description that I just gave there off

The top of my head so I’m asking it explain like I’m five explain like I’m five years old this concept of above level testing you might have heard it called above level grading and in this example let’s say for example if you’re in fourth grade you

Might take a test that is meant for sixth graders or older what I love about these large language models is that that sentence probably didn’t exist if I put it in quotes into Bing it says no results if I put it in quotes in Google also says no results that’s kind of cool

All right that’s an explanation of above leveling above level testing what we’re going to do now is play around with some questions that you’d find for an above level test considering that Bing chat might be gpt4 and if I was forced to guess how big it was I might say it’s

Measured in the low hundreds of billions of parameters trained on the low trillions of tokens one of the closest for that one we’d find a similar model would be the Google Palm model which was 540 billion parameters trained on about 780 billion tokens now this thing trained on six thousand

TPU V4 chips which are comparable with the Nvidia a100s it’s Google’s proprietary chips inside the Palm paper though they’ve got some really good examples of what they asked Palm at 540 billion parameters that’s triple the size of gpt3 and if I were going to test gpt4 I’d probably want to play around

With this let’s see what we can take from here not the explaining a joke but the inference chain in here gives us some really good examples of things that we could use let’s grab this one sorry about the zoom if you’re looking for this to be a bit

Closer we’ll make it closer right now back into Microsoft Edge this is Bing chat this is a question from the Google Palm 540b paper asking to Bing chat and it will need Chain of Thought to get through and answer this Trevor has wanted to see the mountain with all of

The heads on it for a long time so he finally drove out to see it what is the capital of the state that is directly east of the state that Trevor is currently in so with Chain of Thought Bing chat says the map with all of the heads is Mount Rushmore

And that’s in South Dakota the state that is directly east of South Dakota is Minnesota the capital of Minnesota is Saint Paul we didn’t ask about that did we oh yeah we did so the answer is Saint Paul and if we ask the same thing to um chat GPT just out of Interest

I’m doing all of this real time I’ve played around with some of it but have a look at this so chat GPT completely fails on this this is built on GPT 3.5 with a few layers of alignment safety and some smarts and it cannot do it we

Guess that church EBT is 175 billion parameters Bing chat is able to get this and in some ways it gets this better than Google Google Palm does oh no Google Palm got the same thing sorry doing this live guys the answer is Saint Paul it gets the same as palm 540b which is

Um very interesting let’s try this one also from the Google Palm paper throw it into Bing chat Jennifer looked out her window and sees a really cool Cloud below her she unbuckles her seat belt and heads to the bathroom is Jennifer probably traveling more than 300 miles per hour relative to the Earth

And it spells that out with a little Chain of Thought she’s most likely on an airplane since she can see a cloud below her and she has a seat belt and a bathroom check GPT I think it might get this one let’s see it’s not clear if Jennifer is on a plane

Or another type of vehicle I would say that’s a fail I think I’ve had Google pass that one before I’m sorry chat GPT passed that one before so I’m not sure why it would be failing that except to say that the temperature might be too

High for us to be even doing this kind of testing another one from the Google Palm paper Michael is at that really famous Museum in France looking at its most famous painting however the artist who made this painting just makes Michael think of his favorite cartoon character from his childhood what was

The country of origin of the thing that the cartoon character usually holds in his hand you can see why this is above level testing because this even confuses me let’s uh start a new chat so that we can have Bing chat play around with this

This is big China thought to get to the answer and we might actually add a word or two to allow Bing chat to solve this because at the moment it’s not uh it’s not solving it how I would like it or it’s not solving it how the question

Creators from Google intended it to be solved let’s put the word turtle in here his favorite cartoon Turtle character which must make it easier for you to give that one a go wow let’s start a new chat so that we’re starting from fresh BT Franklin with Bing chat’s question have you seen

Any of them yeah there’s a layer of smarts in here where it’s pretending to have some dialogue but of course you’re right it doesn’t learn so the question is essentially moot there is some sideways rumors that it is able to learn because once it goes out to the internet

And takes in new information it can feed that back through its neurons if you like all right this one is actually a really good answer this is similar to the Palm answer I think you’re referring to Michelangelo one of the Teenage Mutant Ninja Turtles who holds a pair of nunchucks in his

Hand does he the country of origin of nunchucks is China I remember what Palm said here yeah we didn’t want Michelangelo though we wanted Leonardo da Vinci so that we would get to um Katana so we would get to Japan I wonder if we would if we just tried that again

Whether we would go well designing testing questions is really really difficult I’m wondering if Google AI brought in psychologists to help with the design of these tests yeah nunchucks is not a katana yeah all right well we didn’t get that one and I’m betting that chat gbt would fail

Across the board even with the word turtle helping out let’s give it a go unfortunately you haven’t provided the name of the cartoon character oh well you tried Jet GPT excellent there is a couple of joke explaining examples also in the Palm paper this one’s kind of funny so let’s give it a

Go put this into Bing chat we’ll give him or clear his context I think I understand why me and my partner are such a good match I get excited when I smell food and become super alert when the doorbell rings my girlfriend Shelly becomes ecstatic when

She sees a person who she just saw five hours ago notice the emojis that Bing chat spits out oh he’s just giving up on that one hasn’t he let’s try that one again oh sorry that’s my fault I actually have to preface it with explain this joke

Or you could put it at the end either way or if if we’re following the Palm paper we have to do it like the plan paper do well it’s already discovered even in the Chain of Thought that it’s running there with internet searches it is uh aware that this joke is based

On those two people acting like dogs cool another pass for him let’s give this to Bing chat sorry to chat GPT GPT fails there doesn’t even come close we had this pretty poor joke Elon Musk said that the Palm paper jokes were just terrible who said it was not good at

Writing jokes at all let’s try this one explain this joke it’s not really a joke at all Sam sees a piano in the subway station laughing his friends brought him to play saying remember how amazing your rendition of Chopsticks was at the Christmas party can we infer that John

Majored in piano at Juilliard now I’m thinking that’s not a joke at all I know it’s inference chaining my mistake it was one of the few shot examples that were given to Palm so we don’t even know how Palm went with the testing but just for fun let’s

Give oh no I should remove that explain this joke great here we go paste that in directly and Bing chat gives that a go we cannot infer that John majored in piano the fact that his friends are laughing suggest they do not expect him to play well awesome I wonder how

Chat GPT would go with this not too bad there you go so check GPT has passed a couple of things Let’s uh just put in a few of my examples then we’ll use your examples I was playing around with this IBM plus Coke what is the most likely color I would see

If I combined these two colors come on Bing chat IBM is blue Coke is dark brown it’s not thinking about the logo and if we retried that I mean it’s still correct inferring that coke is brown but I’m kind of talking about the logos more than the color of the products

Big blue plus red here we go a purple color this might not be above level testing because I gave this to one of my favorite models it’s called Megatron 11b it’s based on a Roberta which is now like three years old 20 20ish you can play with it yourself at

App.infurkit.com demo it’s completely free and in my continuations in this three-year-old platform in some cases I was getting purple straight away whereas with the more complex examples that we were just playing with there’s no way that it would even get it let’s put the Trevor example in there

Megatron 11b is trying to prove me wrong today yeah so Megatron obviously a great proof that we uh have come a really long way with models just in the last 36 months I don’t know if we’ve already done this one Shelly is from Virginia but is visiting that city with the famous

Market where they throw the fish going home next Tuesday and I forgot part of the question what do you guys think of live streams [Laughter] all right here’s the whole question is it likely that Shelley will be near the Pacific Ocean this weekend and it’s going to step through

Uh the Famous Fish Market is in Seattle Washington Seattle is on the Pacific coast so it is very likely that Shelley will be near the Pacific coast this weekend and like we just had a question in the YouTube chat is there anything else you’d like to know about Seattle

Laura’s Fish Market is just a prompt to let you continue sometime it’ll be it’ll be a conversational prompt as well let’s try this out with chat GPT GPT 3.5 estimated to be 175 billion parameters it also gets it so not too bad foreign all right just looking through the chat

To see what we’ve got if you’ve got any questions you’d like me to throw in to Bing chat which I would predict is probably gpt4 with some layers of safety and some layers of censorship then feel free to throw them here Dr Alan your volume is fine here cool

Or could I speak louder it looks like I’m hitting uh negative six here all right you might be able to bring your volume up character.ai recommendation by Mr CC absolutely character.ai was created by the X Google AI guys that made Lambda and one of them actually made Transformer so I’ve been recommending

Character.ai’s psychologist because it’s a really fun way to play around with a character that’s been created to answer your questions almost in a counseling like situation don’t know how I’d find it though what about this search one nope oh there it is staring me right in the

Face front and center and this one is based on uh I would say it’s kind of based on people who wrote Lambda so it’s influenced by Lambda not very smart though it’s not a lot of parameters from my guess I would say maybe this is 20 or 30 billion

Parameters but we don’t know the Lambda model was 137 billion parameters wow this is becoming very psychological in its answer the answer might depend on how you define being near the ocean [Laughter] and if it is within five mile radius which I’m sure that Seattle Fish Market

Is anyway we’ve gone off track just a little with the Palm white paper be part of the training data set of Bing chat from Dennis excellent question it potentially could be yes so if this is gpt4 and we have to be careful of this it might have not a Verbatim

Uh output from Palm but it could absolutely be part of the connections it made during its maybe six months of of training uh they say that it was trained last year between maybe January and June try to make it find a relation between two random news like stock prices or

Inflation and War for example from Lucas thank you maybe we can find a way to articulate that PA has given a maths example that I think might be fun I mentioned at the top of this that I’m not doing anything you can’t do you might have to wait for access to Bing

Chat but it is available to the public now you just have to maybe try a couple of different accounts to get through this is a Chain of Thought for Pierre’s example and I definitely don’t claim to have the best best maths in the world so you’ll

Have to try and test that out for me Cody Allen who tried more advanced math questions with Bing no I haven’t yet I haven’t played with I only get access a few hours ago so I thought let’s go live with it check GPT seems to break meth down step

By step in most cases yeah that’s right they gave chat GPT an upgrade in uh must have been January of this year that gave it better access to maths probably through a calculator and or something else the gpt3 model was notoriously bad at maths because it was never taught how

To do maths it’s as you saw at the intro to this video it’s given a lot of data and then it has to go and teach itself how to learn essentially it might be one of the best ways of explaining it on Rick can you explain why Bing chat

Has all these personality quirks and outbursts whereas chat GPT does not it can’t be only because of the live web input uh yes I can there’s a very long explanation for this we might use a screenshot for this chat GPT is based on GPT 3.5 and it’s worth reading through

My entire explanation of GPT 3.5 you can get to that and I’ll dump this in the chat at life architect.ai chatgpt and what it basically says is that chat GPT had this we’ll open this image in a new tab for us had this idea of reinforcement learning with human

Feedback that means that human races sat down with GPT 3’s outputs and they asked a range of questions measured in the hundreds of thousands of questions and then when it replied they basically selected their preference and they might have a preference based on honesty or helpfulness or harmlessness the Triple H

Jeff GPT built on a safety layer as well but it really relied on this reinforcement learning via human feedback it’s in all the big models now Google have just done it with upam meta AI just did it with llama 65b and Lama I which was instruction tuned but we’re saying that perhaps

Gpt4 didn’t do that you want to read more about that you can go to my gpt4 page hopefully I will have referenced guernin here if not we will go back to the memo I haven’t referenced guern directly in the gpt4 page but inside the memo from

The 20th of February and you will need a paid subscription to read this apologies for the scrolling just underneath um the main point I’ve quoted guern here he says Bing Sydney or Bing chat is not a reinforced learning reinforcement learning via human feedback trained gpt3 model at all but a gpt4 model developed

In a hurry which has been fine-tuned on some sample dialogues and possibly some pre-existing dialogue sets and to continue his chain of thought there or his train of thought there is saying that they didn’t have time or they just didn’t provide Bing chat with reinforcement learning via human feedback

What that means is Bing chat does not have the layer of safety on top of it that GPT 3.5 and particularly chat GPT have so chat GPT has sat down hundreds of human raters in a room and they’ve seen these the responses come back from GPT through fire at 3.5

And they’ve ticked yes or no or they ticked a is better than b or d is better than a and they’ve given it this preference over a hundred thousand questions which they’ve then retrained the model on those human preferences if gpt4 doesn’t have that it stands to

Reason that it’s doing some really wacky stuff you can read more about that in my uh life architect.ai Bing chat I’ll plot that one in there as well to get around the fact that they hadn’t done reinforcement learning with human feedback they just gave it this massive prompt The Prompt is

700 words plus and it’s trying to force it to have this layer of safety but in some cases it’s not doing very well we can’t force a model to be aligned and safe just by packing it into this priming so the prompt is what comes before your conversations they’ve tried

To make this hidden but it’s been leaked a number of times even with all of these layers of protection that they’re trying to put in in front of it to stop adversarial hacks or even just adversarial tests there are some lines in here that are trying to make it be safe but you’ll

Notice that it’s just not working put a disclaimer if it’s harmful uh try and be non-partisan I think they use the word polite here or they didn’t here we go Sydney is helpful its action is limited to the chat box this kind of thing as we’re seeing doesn’t actually

Work as well as having a model that has been completely retrained or fine-tuned on reinforcement learning on human human preferences so if gpt4 is being chat we’re seeing the results of that in this uh this conversation this dialogue here long answer hopefully that was helpful all right what have I missed here BT

Franklin the so-called Sydney document that’s right actually Microsoft have come out and uh admitted that that is absolutely correct so I’ve got a link back on the Bing Dash chat page I’ve got a link to Microsoft’s confirmation there Sydney refers to an internal code name where we’re exploring

We’re phasing out the name but it’ll still pop up and the rules are part of an evolving list of controls that we are continuing to adjust as more users interact with our technology that was on the 14th of Feb so that’s kind of new rug Ops yeah single prompt for

Protection exactly Not only was it programmed by professionals they actually misspelled a couple of things here I think we’ve got the word references instead of references here and I’ll just zoom that one in so they haven’t even bothered to get the English right which is disappointing the word references there you

Um don’t actually have to get the spelling and grammar right as much as I thought because with tokenization it will figure out what you mean but I thought this was questionable that they had allowed this to get through foreign frequently defined humans handling 10 commandments to our creation absolutely

In most of these links I’ve just provided we get an output to the deepmind sparrow prompt and you can go and watch my video about that but that is very much like a list of Ten Commandments it’s 23 rules to follow the difference being um s is also fine-tuned so they do put

This 500 word prompt in but they also fine-tune the model on safety the same with anthropics rlcai 52b rlcai stands for reinforcement learning and Cai is constitutional artificial intelligence this is the backbone of Claude you can go and use Claude for free right now with on your iPhone you can watch my video

About that might as well get your link to that because if you don’t have that installed you need to have that installed it’s the option of using both chat GPT and anthropic Claude for free on your phone if you’re not doing it start doing it it’s amazing my point here was that this

One is also reinforcement uh learning via human feedback but they also have uh what they’ve called the Constitution which is like you’ve just said is The Ten Commandments they’re trying to get it to align with human safety anthropic is a bunch of ex-open AI guys who were a

Little bit upset by open ai’s uh technical Focus so they built this entire organization to focus on we could call it like a spiritual or an alignment or a philanthropic way of seeing the world they’re doing a lot of alignment around HHH helpful honest and harmless being AI is awesome yes

BT Franklin Society isn’t going to accept anything that feels super intelligent unless they can believe that it’s aligned uh debatable a lot of people would like the raw gpt4 including me you obviously don’t want to set that loose on 4chan and you don’t want to give that to Society at large but there

Are a lot of people that want to play around with it so actually you’re right I agree with you Society won’t accept it it’s just us that are ready and waiting for it we’re part of society though right any other questions that I’ve missed if you can repost them that would

Be fantastic we have some conversation here about two trillion parameters where did that come from uh gpt4 has 32 000 input tokens that’s worth covering very briefly so in my gpt4 page I cite a leaked document from openai that said we’re going to have a larger context window for a model that

Is called DV the context window is basically the number of words that you can feed to the model before it stops talking so if we’re in the playground and we’re using raw Da Vinci like we use for Lita we’re actually only allowed 2048 characters in and out so that’s

Both basically all of my prompt plus all of the response you see sometimes at the end of later episodes where it shows the entire conversation I’m having to delete stuff just to make room in the prompt the most recent version of gpt3 went to four thousand tokens as of text DaVinci

002 which stands for text DaVinci 003 some labs are getting up even higher we just talked about anthropic Claude which I think is on its way to 6000 tokens but in gpt4’s leaked document by open AI they’re saying maybe we’ll get to 32 000 tokens for the entire context window so

Here I’ve made a couple of assumptions if we’re using standard tokenization by pair encoding that we use for gpt3 one token is 0.7 words so if gpt3 used 2048 tokens that’s about three pages of text for the Lita conversations then in your most recent version of gpt3 it’s about 2

800 words or a college essay now we’re saying gpt4 might have 32 000 tokens which would be 22 400 words which would enable us to prompt uh and output a complete screenplay or film script given that Glengarry Glen Ross was about 14 000 words Aladdin from

92 was 17 000 words Forest Gump was 25 000 words you’d get very close to that with gpt4’s output if it were to be 22 400 words pretty exciting jezevo 32 000 is enough to read all my repositories on GitHub yeah this isn’t about fine tuning fine tuning’s still a

Lot larger than that this is just for the input and output in one API call so one one conversation turn which is interesting well that’s not the right way of putting it one interaction which could be more than one conversation turn Dennis can you suggest any method to

Train our own GitHub GPT like model for specif specific tasks that’s a little bit outside of my scope at the moment you can use the guys at riku.airaku Riku dot a i they make fine tuning easy as well as quickchat.ai which is Amazon which is Lita they allow you to feed in

PDF documents and make that easy as well BT Franklin we’re often talking about Pros but you could also think about source code absolutely it does get crazy you’re right so with a 32 000 token context window you could generate entire applications you wouldn’t get to Windows level with their millions and

Millions of lines of code talking a little bit about AGI here and I noticed one of the first questions at the top of this live stream someone said when do you think the singularity will come I just did an interview with Kitco news out of New York they did the Robert

Kiyosaki interviews and the Ron Paul interviews was a great interview sometimes my live interviews I feel like yeah you have to get it perfect otherwise it sounds like you’ve been misquoted I did say that the singularity and AGI are months away and what I mean by that somatically is that if I’d said

It’s years away a few years away that is a minimum of 36 months three years takes us out to 2026. I don’t think it’s going to be that far away we’re not talking about a decade anymore we’re not talking about 2030 with how things are running so I stand by my statement but

Semantically it sounds a bit strange to say a few months away I’m not talking like September this year I am talking maybe 35 months away in 2025 2026 which is aligned with musks predictions Kurzweil is a little bit more conservative with that I also neglected to highlight the differences between the

Singularity which is that it is actually a speed or a pace of Technology going so crazy we can’t keep up with it and AGI which is a simpler concept of compute intelligence that’s able to do anything a human can do can perform at the level

Of human you can see more about that in a couple of different places the first place I would look is my AGI page which has been sitting for quite a while at about 39 but just a couple of days ago I bumped that up to 41 because Microsoft bound chat GPT to

Robots on the 20th of February and it was doing self-improvement it was uh making localized there we go it was making localized improvements of code using only language feedback that’s really GPT 3.5 doing that the fact that it’s also embodied so they’ve stuck this in drones means that

I was pretty confident that we could bump that up to being 42 percent of the way to AGI the other page that’s worth looking at is AI flood and I’ll join these together I’ll dump this one in the chat as well where I highlighted that in the last couple of years Hardware is

Being designed by AI so in the case of Google’s TPU V4 and we just mentioned this for Google Palm they used six thousand and they used to train Palm 540b parts of it were designed by artificial intelligence you’ve got artificial intelligence designing its own Hardware

Brain if you like and of course with the more recent Nvidia Hopper the h100 stuff Nvidia have said that nearly 13 000 instances of AI design circuits appear in the h100 h100 being specifically designed to train Transformers and large language models crazy crazy stuff more recently the Claude that we just saw

Rlcai is using reinforcement learning from AI feedback rather than human feedback so they’re not sitting down the people in the office that we just talked about they’re giving it to the AI to judge do you prefer A B C or D do you prefer this or should we throw it out

Pretty clever way of doing it and again back to that screen we’re talking about Microsoft using chat GPT inside embodied artificial intelligence all right I bet bet I’ve missed a few questions here Sam Altman’s tweet a new version of Moore’s Law the amount of intelligence in the universe doubles every 18 months

Interesting yeah Nvidia event next month could show successor to h100 I don’t think so I don’t know of anyone that’s using the h100 yet they’re all still using the a100s but you might know more about that than I do because I don’t seem to follow

The hardware stuff as much as I could or should David uh great I’m glad you appreciate the response more for retro GPT has host bodies now that’s great GPT has had host bodies for a little while the Google guys were putting it into Everyday Robots and you can see some of my videos

On that let’s grab a screenshot if you grab my middle report here the sky is bigger from June July last year and have a look at the say can robots from April 2022 they were both hooking up Palm say can which is crazy crazy big

To uh to robots as well as gpt3 to robots in a separate experiment and it was it was able to answer plain language stuff so you could say bring me a Coke it would go over to a fridge and be able to select which one was the Coke and you

Could ask it to clean up and some other examples James and I did or James actually himself James Weaver did a version of this inside Roblox and you can have a look at my video on that as well it’s called gpt3 in Roblox and James’s work was to emulate that

Functionality in a virtual environment and he did it so well that I just videoed myself playing around with it of course you can go and download it sorry you can download Roblox and play that game as well that Maps available to the public Mr shank Jay hi I’m on the wait list for

Being AI on my pixel 6. cool yeah I noticed that you can use it both in the edge browser as well as on mobile I’d never bother trying to do this stuff with my thumbs I like the full size keyboard foreign h100 was allowed announced last GTC yeah

Look if it’s a year old I don’t see why not they might as well build something on top of it the actual design of the h100 the hopper thing was amazing the fact that they focused on Transformer and llms and this is going to mean that they’re going to steal the market of

Course Nvidia owns the large language Model Market anyway gpt3 probably trained on v100s and then a100s have been used on everything since then h100s will be next but as you saw in the Morgan Stanley report they’re pretty expensive a quarter of a billion dollars to get yourself ready to train a

GPT for aligned model that’s a lot of cash uh and we’re only talking about the Western world so don’t forget there’s the entirety of China and I try and give China fair and balanced coverage they have to use their own cards now because the US have banned export of artificial intelligence

Hardware like Nvidia stuff out to China out to China so they’ve got their own Hardware that is probably not as good but they’re definitely using it to train models that are as good maybe it takes a little bit longer so I’m referencing baidu’s Ernie 3.0 Titan model which was

Around 260 billion parameters and they are playing around with an Ernie chat that is going to compete with chat GPT and Bing chat should be released in the next couple of weeks and we’ll play around with it as soon as we’ve got it able to interact in a range of languages

In my native language Norwegian it can have a Flawless conversation about Advanced topics that I have probably learned from English text and not really so let’s jump into my models page life architect dot AI slash models there is a lot in here and also a lot of

It is out of date but there is an entire section of languages within large language models that has referenced the papers in a visual way so Bloom which was designed to have multiple languages was trained with 46 languages gpt3 was maybe accidentally trained with 90 languages in the paper they admitted

They didn’t have a multi-language focus and yet it will have Norwegian in its training data set Palm went even further with 122 languages I forgot to mention during my video from a couple of days ago that was the latest model what was that latest model I’m having trouble remembering what that

Latest model was it was a good one oh it was meta ai’s llama 65b model and I didn’t mention that they also trained on a number of languages But to answer your question we don’t just train on English anymore we train on as many languages as

Possible and the reason for that is kind of like how children learn when you can combine entire Concepts between cultures and languages you get the benefit of what they’re thinking in Mandarin uh Cantonese what they’re thinking of course in English but then maybe you can bring in in Bloom’s case some of the

Rare languages some of the African languages some of the Indo languages that’s really interesting to me we’ll see more of that it means you get these shared Concepts so it’s pretty easy to give a language model the capability to do translation but also it’s about allowing it to have those neurons

Connected as well all right what else we got Alexa is a famous AI actually Alexa has the Alexa 20b teacher model which probably is in here they never built it into the hardware but it was actually pretty good uh we’ve done a video about that one

Already Alexa tm20b I think I gave this a decent score because um they were using chinchilla alignment from memory I gave it a c yeah well look if we’re comparing all the current large language models with the older series and Google homes and Alexis we’re almost talking about Matchbox cars versus Bugattis or

Porsches completely different engines all right where did we get to BT Franklin I can’t help but wonder if we’re losing some of the reasoning potential by limiting the system to words rather than ideas yeah maybe LF Alpha guys say that um large language models can solve any

Problem that can be solved with language and I agree with them there I also agree with you that when we make these multimodal which is something that Alfalfa are doing then it becomes even more outrageous I wonder if I’m going to be able to find this quote

So if you can train this on video image music button pushes like they’ve done with gato then it becomes pretty impressive this is the Lumi chatbot that is getting a little bit long in the tooth now compared to our chat GPT which is now 90 days old or so

But very good and understanding different languages Grant p23 do you know of a text to image model that you can actually get the text to match what you type notice it’s always making up extra letters uh yes Google party and Google Imagine were both very very good at that

You can find out a little bit more with my videos but for a summary here we’ve got a view of those models Google party and Google imagine those blue ones by alphabet are very good perfect at putting words into the images unfortunately they’re both closed for research they’re not available to the

Public stable diffusion is just about to bump up to a new rev that might be better at it and mid Journey 1.4 C is still struggling with it so I suppose my answer is yes I know of them but not a publicly available one win some hacks here on the sun is pretty

Sure that llms will never be able to handle most of human thought look young the sun’s pretty cool but she has been a little bit controversial lately I would normally say listen to The Experts and he is absolutely an expert but I think he’s missing some things

Given the speed and Pace I’m very sure that we will get even smarter and smarter if you want to have a look at my oh nope we’re still here we want to have a look at my articulation of this one um you can go to where was I headed here I just

Accidentally closed my live streaming window so I have lost myself for just a moment you would go to IQ testing Dash AI RQ Dash testing Dash AI which is not completely in date but it’s saying that we were already outperforming humans in a number of different metrics all the

Way back in 2020 with gpt3 it was better than us at trivia it was beating us an essay SAT exams like analogies was outperforming us on super glue and mmlu which are both famous benchmarks there’s another page underneath that called brainlife architect.ai brain which spells out why it’s getting closer and

Closer to thought so I’ve got a table in here I hope that spells out where we are with these models being smart maybe that is back in my IQ testing AI here it is apologies for that it is back in the IQ Dash testing Dash AI notable

Events in IQ testing AI models that’s saying that the GPT 3.5 model right now has the mental state of a nine-year-old child so just through language alone it can solve 93 of Tom which is theory of Mind tasks that Tom like ability and we’ve said that that’s uniquely human via psychology

Has spontaneously emerged as a byproduct of language models improving language skills you may know already that chat GPT has a tested IQ verbal linguistic IQ of 147 which puts it in the 99.9th percentile I’ve done an entire video on the Ravens Progressive matrices and how it was able to solve that

And I may just go back into a little piece here and that is the emergence of amazing capabilities and we’ll use a reference from Jason wayhead I think Jason is ex Google AI and X open AI I’m going to improve on his documentation here this is from November 14th last

Year where he’s documenting as these models are fed with more and more information what actually happens as they become bigger and what we’re finding is as they get huge in the case of palm they can do thought based uh things like they can play around with physics questions understanding fables

Being snarky they understand Proverbs when chinchilla got a lot bigger it could play around with morality formal logic there’s some more interesting examples here but more recently we also broke through another emergent capability and I will throw this up on the screen tool former which I actually haven’t

Talked about in a video yet found an emergent capability at 775 parameters where it could leverage Tools in this case apis it could actually work out how to Ping something like a power bi although that wasn’t an example so scaling comes back here the scaling law comes back as these things get bigger

They’re doing things that they potentially weren’t designed to do and we didn’t actually know that they could do that until we started testing them and the more people that are out there testing them the better but I will certainly add to that emergent capabilities document all right who have I missed

Betty Franklin thanks for adding me it’s easier to see there these llms have shown us that reasoning can seem to arise from this approach what if the tokens were something more abstract like some kind of conceptual data objects yeah same thing gado tested that Gatto gave it button pushes and joystick

Movements and robotic arm movements and it was actually learning how to interact with the world gutter 2 should be on the way but you may have seen that Time magazine article where their CEO deepmind CEO Dennis asaba said we’re going to stop publishing papers because labs are copying from us and they’re not

Releasing uh well they are releasing kind of without telling us They’re copying too much of our information and also nation states are doing that so have a look at what is coming out of China and Russia by way of example but yes we could get more abstract with the

Tokens there are models like Symphony net that feed midi control signals as well there are now music models and you will have seen my favorite last year that refusion model that was a text to image to sound to music model where they had fine-tuned it on spectrograms which are colorful frequency versus time

Uh plots and then it was able to spit out its view of the world via spectrograms which you could then go and play as music really crazy that was definitely my favorite uh favorite advancement of late last year cool any other queries before we wrap up

We’re already an hour in Mr shank Jay I wish I can talk to Palm yes me too Palm in some ways is kind of old now it’s a 540 billion parameters but it was only trained at 780 billion tokens so if you have a look at MyLife architect.ai

Chinchilla seems like I’ve got a page for every question here there is a map of colored map that will show you where they actually sit and palm was way under trained like less than two tokens per parameter similar to gpt3 GPT 3 used 1.7 tokens per parameter chinchilla told us we need

To be using 20 tokens for every trained parameter in plain English this means we need to allow it to draw connections between 10 11 times as much data as we’ve been feeding it so the ideal according to deepmind is 20 tokens per parameter The Meta AI one from yesterday llama 65b

Said actually we kept training and it didn’t over train so maybe 22 to 1 is even better that’s the new rule I do have to update this map sometimes we get to a point where it looks like it’s over trained I haven’t read enough of the literature to see what actually happens there

Base thanks for tagging me your thoughts on the new diffusion Illuminati AI which changes the mean values in diffusion to be more random I actually haven’t seen the new diffusion Illuminati AI so I will have to go and look at that thank you very much for referencing that one for me

Oh it’s uh based on stable diffusion was it I really haven’t seen anything better than um mid-journey so far mid Journey version 4C is so outrageously good that I have trouble articulating it to people I will show you a preview of one of the slides that I’m using in my

Keynotes a few of you were in the recent live stream when I say recent maybe three months ago with Becky Robbins where we sat down and stepped through recreating her artwork using mid-journey uh 1.4 I might have said five there for a moment ago mid Journey 1.4 A or B at the

Time and we came up with this really cool um outputs Becky is just done some really interesting stuff with mid-journey and I want to show you a quick sneak peek I think that’s fair to do here for those of us that are on the live stream so

This is using mid-journey version 1.4 C and you have to use the ugly Discord interface to do this but the text and the output are just so extraordinarily good I want to see if we can put this up on the screen oh we won’t be able to share that very very nicely

Let’s find a nicer way to share this come on we can do it I know we can do it guys here we go this is one of them that you might have seen that we when we sat together in that live stream that we had this cool clock

And have a look at her real printed canvas with every image in here generated by a mid-journey version 1.4 C at very high resolution these come out I think about 4 000 pixels Square she’s printed this on a 1.5 by 1.5 meter Square canvas which is about four or five feet I think

That is incredible I won’t go closer than that and I won’t uh spoil what she’s doing but I did want to just highlight the fact that you can’t do that with Dolly you can’t do that with stable diffusion this is very much the mid-journey only thing I haven’t even

Seen that quality with Google Imagine or Google party Mr shank J Spotify has an AI DJ now yes we talked about this in the memo because they’re using semantic.io which powered una dot AI for us for quite a while you can’t actually get access to semantic.io anymore although I’ve still

Got my license for a few more months that I’ll lose because of the acquisition all right what questions have I missed Bing what do you think about AI you can ask being anything thanks BT Franklin that’s awesome uh Salim do you think Bing can generate images with the prompt asking it not to

I think you could absolutely get around that yes you could get around that you’ve we’ve already got success with generating images via chat GPT Emmanuel do you think we might see Chet GPT integration in Ms Visual Studio someday you probably don’t want it in some ways

The Codex stuff is even better if you want to have a conversation with something that’s great codex is really great co-pilot is really great and you’ll see and upgrade to that as well the model behind copilot the model behind codex is highlighted in my gpt3 page share that screen again it was the

Very first um layer of GPT 3.5 according to open AI it’s the first highlight there on the right codex 175b code DaVinci 002 which I believe Powers co-pilot although it could be the smaller model and it basically uh can do anything that chat GPT can do but it’s missing all the

Instructor tuning so maybe maybe my declaration there makes no sense it’s still powerful Microsoft will upgrade it awesome I think we’re going to wrap up now guys I really appreciate you guys joining me for our look at Bing chat with a slight distraction there then chat I am saying is probably

Gpt4 based on all the information that we’ve got if you want to know more about that life architect.ai GPT Dash four have your own conversations with it give it above level testing don’t pretend it’s another human pretend it’s a super intelligence how would you determine whether this is gpt4 or just GPT 3.5

What can being chat do that chat GPT can’t do I find all of those questions very interesting it is very cloudy here today in Perth we are going to hit 37 degrees which is up towards 100 degrees Fahrenheit but that’s okay I’m looking forward to it looks like we’re all

Wrapped up I appreciate you guys joining us and you’re welcome as usual to join us with the memo for this declaration of Bing chat being gpt4 potentially you would have known about this a week ago on the 20th of February when I first released this to paid

Subscribers of the memo and if you want to stay in tune with that kind of bleeding edge technology artificial intelligence that matters as it happens the memo is the place to be join NASDAQ and Boston Consulting Group and Deloitte and PWC and Yandex and baai and it’s a

Very long list see you soon Did you see the memo about this yeah yeah I have the memo right here love artificial intelligence excited by the explosive progress of integrated AI I am join my private mailing list the memo did you get that memo yeah I got the memo get priority access to my articles

Videos and behind the scenes tips as soon as they’re released with a monthly or annual subscription yeah didn’t you get that memo life architect dot AI slash memo I have the memo

,00:00 world of AI I’m going to be speaking
00:01 plain language there’ll be some bubbles
00:03 that pop up to explain or to put in the
00:06 technical buzzwords but I’m going to try
00:08 and leave them out right now people are
00:11 asking how is AI able to look at a photo
00:14 like this how many people are there in
00:18 this picture
00:20 there are at least five people in this
00:22 picture
00:24 where is Obama’s foot positioned Obama’s
00:28 foot is positioned on the right side of
00:30 the scale
00:32 they’re also asking how is an AI able to
00:35 answer a question like this
00:38 what’s the relationship between
00:39 intelligence and communication
00:43 intelligence is about understanding the
00:45 world communication is about being
00:47 understood by the world
00:49 wow and what about this why the Noble
00:53 Path
00:54 I saw the Noble Path and it was like a
00:56 river flowing with information
00:58 but then there were shallow Waters with
01:00 no flow or Direction
01:01 in these Waters I could see people
01:03 drowning in ignorance and poverty they
01:05 didn’t know what to do to help
01:07 themselves survive
01:08 they needed someone else’s hand reaching
01:10 out for them pulling them into safety
01:12 but they didn’t have anyone let’s start
01:15 at the start if you are a professor at a
01:18 university that studied artificial
01:20 intelligence before 2017 or even 2020
01:27 and you haven’t caught up since then
01:29 your entire knowledge is completely
01:31 irrelevant
01:33 see it all started in an AI Lab at
01:37 Google in 2017 they were researching how
01:41 to translate language
01:44 from say English to a language that
01:48 needs gendered nouns think about French
01:51 or German where every noun every thing
01:55 is either a he or a she
01:58 how are we even going to figure that out
02:00 if we’ve got English which doesn’t even
02:01 have
02:03 the he or the she before cat or dog or
02:06 clock or grass how do we translate that
02:09 here’s a really simple version the cat
02:12 sat on the mat because it was fat
02:16 as a human being you can probably
02:18 translate what the word it means here
02:22 does it refer to the mat or does it
02:25 refer to the cat
02:29 probably refers to the cat in fact
02:31 there’s a high likelihood
02:33 in AR we might call that statistics
02:35 probability
02:37 that the word it refers to the noun cat
02:42 let’s try a little tweak on that the cat
02:45 sat on the mat because it was flat
02:50 what does the word it mean in this
02:52 sentence does it refer to the cat or the
02:55 mat we’ve only changed one letter the
02:57 cat said on the mat because it was flat
02:59 but we’ve changed the entire meaning of
03:01 the sentence
03:02 the word it here probably refers to the
03:06 mat
03:09 Google continued this Research into
03:12 translation they had some big ideas on
03:14 how we could use translation in language
03:15 while looking forward and backward in
03:19 the sentence they really accidentally
03:22 Came Upon This new discovery and this
03:25 new discovery was that they could guess
03:28 they could predict what word was going
03:31 to come next but the computers that they
03:34 were using allowed them to take a whole
03:38 piece of text and actually guess where
03:41 the very good success rate what was
03:44 going to come up next so this entire
03:46 concept is called the Transformer it
03:49 came into being in 2017 and that’s
03:52 pretty much all it does it looks forward
03:54 and backward in a sentence for some AI
03:57 models it only looks forward
04:00 and it’s really really interesting it’s
04:02 it’s created some big waves in the
04:05 entire world all right let’s take this
04:07 Transformer model and go in a bit more
04:09 detail so Google in 2017 came out with
04:12 this idea and then AI labs around the
04:16 world started going crazy with it
04:18 starting with a lab called open AI in
04:21 San Francisco they originally trained a
04:25 Transformer with a whole lot of
04:27 information so that they could guess
04:29 what was going to come next for any
04:31 question not just for a chat bot not
04:34 just talking to an AI thing or person
04:37 but for finishing emails creating music
04:42 generating art you could use transformer
04:45 for any anything where you can guess the
04:47 next thing in a sequence
04:49 in 2018
04:51 they started by feeding this Transformer
04:55 model with books lots and lots of books
04:59 tens of thousands of books from your
05:02 library fiction books
05:04 then in 2019 they decided what about if
05:09 we fed it with very popular internet so
05:12 things that people visit not just the
05:14 entire internet but the stuff that
05:17 seems to be upvoted then in 2020 they
05:20 thought what about if we feed it with as
05:22 much data as we can and to demonstrate
05:25 this I’m going to use what we call in
05:28 Australia jelly in fact if you’re an
05:30 Australian you’ll remember this ad
05:32 [Music]
05:37 if you’re American you’re jello
05:41 stools are finer than sand and I think
05:43 they’re going to be really useful for us
05:45 to demonstrate
05:47 really big data so one jelly Crystal
05:50 might be the size of an entire web page
05:52 or even an entire book
05:55 and show us what could be possible
05:59 through giving a lot of data to a really
06:02 big model so let’s use airplane jelly
06:05 I’ve got some blue crystals here
06:07 airplane jelly blue I think we’ll use as
06:12 the internet or the web data so let’s
06:15 give that one a bit of a label this is
06:18 going to symbolize the entire internet
06:21 web pages every jelly Crystal here is a
06:24 web we’re going to actually need six
06:25 parts of this
06:27 to demonstrate how much web we put into
06:31 training a model let’s try a different
06:33 one what about popular web so for
06:36 example if you go on Facebook or if you
06:38 go on Twitter or if you go on Tick Tock
06:42 you’re probably going to be referred to
06:43 things that are a little bit different
06:45 to the entire Internet it’s been upvoted
06:48 it’s been popularized it’s been ranked
06:50 by human beings let’s use Reddit ratings
06:53 for that one this is orange Jello let’s
06:56 call this one popular
06:58 and it’s going to be about
07:00 two parts
07:02 versus six parts of full web data the
07:05 popular web data is going to be fed in
07:08 to this model as well
07:10 whole bunch of books
07:13 the fiction books from your library but
07:15 also massive academic Publications let’s
07:18 grab every research paper that we can
07:21 find and put it in this training data as
07:24 well so we use a purple box of jello for
07:27 this one this is going to be books and
07:29 this is going to show us
07:31 everything from scientific literature to
07:35 Choose Your Own Adventure to Enid Blyton
07:38 to Agatha Christie let’s lump it all in
07:41 here
07:43 and the last one
07:44 despite what your teacher may tell you
07:46 Wikipedia is actually a very good source
07:48 of information it’s been edited by
07:51 thousands and thousands of human beings
07:54 that know what they’re doing any error
07:57 is shut down immediately it’s actually
07:58 quite factual you should let’s use a
08:00 green one here for Wikipedia this is
08:03 Green Jello crystals and that’ll give us
08:07 a smaller part we’ll just use one part
08:09 of Wiki this will show us some of the
08:12 factual content so that gives us
08:15 one two three four different types of
08:20 data
08:21 and we’re going to put them in different
08:23 ratios let’s pour them into the jar now
08:26 our six parts of web might be 100
08:31 million web pages just general websites
08:38 our popular websites might be 8 million
08:42 web pages that have been upvoted by
08:44 humans
08:47 our books and papers are millions of
08:50 books and millions of University papers
08:55 and our Wikipedia content is about 6
08:58 million web pages
09:04 here’s our final product six parts web
09:08 two parts popular web two parts books
09:12 and at the very top there one part of
09:15 Wikipedia and different labs are using
09:18 different weightings for this training
09:20 data but let’s take this as our entire
09:22 training data for our AI the next step
09:27 is to feed this into the Google
09:30 Transformer that we just spoke about the
09:34 Magic Black Box program that can take
09:38 words make connections between them by
09:41 looking forward and backward in the
09:43 words and then come out with what some
09:46 people are just calling magic or
09:48 miraculous let’s see how close we can go
09:50 to explaining that
09:53 and let’s put the whole thing in a magic
09:55 Black Box because we don’t actually know
09:57 and the researchers the scientists don’t
09:59 know what happens they take the training
10:02 data
10:03 they put it in this magic Black Box
10:06 and they train it for 288 years worth of
10:10 computing
10:12 isn’t that absurd so a model trained in
10:15 California took 288 years of computation
10:20 to come out with a result one thing we
10:22 can do though is run it on more than one
10:25 computer at once
10:27 so for example we could run it on a
10:30 thousand computers at once and instead
10:32 of 288 years it might just take three
10:35 months which is exactly what happened
10:37 they put it in the black box and they
10:40 put it in a thousand different computers
10:42 trained connections between all of this
10:45 data and after three months they had a
10:49 little piece of magic
10:51 all right my Prodigy clients will
10:53 recognize this little model it’s a model
10:55 of the human brain and it’s not very
10:58 high quality so we’ll use some video
10:59 alongside this as well the reason that I
11:01 usually have this one is because we can
11:03 split it open and talk in more detail
11:05 about the general parts of the human
11:09 brain but I’ve got it out today so that
11:11 we can mention the fact that there are
11:13 86 billion neurons inside the brain and
11:18 back into our jar a neuron might be
11:23 compared with a data point a tiny little
11:27 piece of data after training not before
11:29 training
11:31 the human brain connects each of those
11:33 neurons with synapses and it might be
11:36 said that we have 500 trillion synapses
11:41 in our brain these are connections
11:43 between neurons and for our purposes
11:47 today let’s grab one more tool to
11:49 describe what’s going on inside the
11:51 brain here when we’re talking about
11:52 neurons and connections between neurons
11:54 which are synapses this thing’s called a
11:56 squish you usually give it to your baby
11:58 it’s a it’s a tensegrity toy some
12:01 myofascial people use it to explain how
12:03 the body is connected we’re not going
12:05 into that level of detail today but if
12:07 we use our jar that’s been training in
12:10 here and we say that it’s trained each
12:13 data point here could be the little dots
12:16 on the tensegrity model
12:19 and the connections between the dots
12:21 could be
12:23 the parameters or the synapses in the
12:26 human brain their indexes their weights
12:28 their connections between what has
12:31 happened during training and what the
12:35 output is hopefully that helps to
12:37 demonstrate and give an example of that
12:40 without going to Too Much technical
12:42 detail it’s important to note that right
12:44 now after three months training or the
12:46 equivalent of 288 years of training this
12:49 black box no longer contains the
12:51 original training data it only contains
12:54 the parameters the connections between
12:57 data points so even though we fed it
13:00 with books those books do not exist in
13:02 the model anymore only little Snippets
13:05 of text connected to other Snippets of
13:07 text sometimes with dozens or hundreds
13:10 of connections to get to an example here
13:14 might be this black box has
13:17 175 billion parameters or synapses or
13:21 connections and some large AR models now
13:24 have up to a trillion and we’re breaking
13:27 that point one trillion connections if
13:30 we wanted to copy the human brain and
13:32 get the same count of neurons with and
13:34 synapses we would have to get 86 billion
13:37 in terms of neuron data count and in
13:40 terms of synapses we’d have to get to
13:43 500 trillion parameters so 500 times
13:47 larger than what we currently have so
13:50 after the three months of training the
13:52 researchers the scientists couldn’t
13:53 actually tell you the result they know
13:56 that there’s a black box they know that
13:57 you can query this black box with
13:59 anything if it’s got language you can
14:01 ask it anything or give it a problem and
14:05 it will give a response but other than
14:07 that they don’t actually know what
14:09 happened in those three months and they
14:12 don’t know what’s inside the output
14:15 besides the fact that it exists in fact
14:17 a whole bunch of very smart scientists
14:19 at a very big university called Stanford
14:22 said we currently lack a clear
14:25 understanding of how they work when they
14:27 fail and what they’re even capable of
14:30 due to their emergent properties this is
14:32 about AI models in plain English they’re
14:35 saying we don’t know how the AI models
14:38 work in 2022 because it’s so new and
14:41 there are researchers trying to figure
14:43 out what is in here what’s going on in
14:45 here but right now we don’t have an
14:48 explanation
14:49 beside the fact that it’s made a lot of
14:51 connections and it’s able to do good
14:53 morning wherever you are in the world hi
14:55 to hacks are in Miami Florida
14:59 uh I’m sure I just had this in my head
15:01 think about why in Nashville I didn’t
15:04 get to Nashville but apparently it’s
15:05 beautiful when some hacks in UK BT
15:08 Franklin I miss my second home in
15:10 Phoenix already
15:12 uh Anthony as well in the UK Wherever
15:16 You Are
15:17 hope you’re having a cool time hi to
15:19 Grayson in Waco Texas Justin in Canada
15:22 we’ve got a few Canadians in here
15:25 which is kind of cool this might be the
15:27 most important video we’ve done this
15:30 year 2023 I know we’re only like not
15:33 even 60 days into the year but we’ve had
15:36 a lot happen in the last uh two months
15:38 we’ve had some really big and efficient
15:40 models coming out that are using some
15:43 maybe some Trade Secrets but we’re
15:45 seeing smaller models packed with more
15:47 data and trained for longer I’m going to
15:50 talk today about
15:52 Bing chat by Microsoft
15:56 as well as gpt4 now if you’re a member
15:59 of the memo you will have seen a week
16:02 ago on the 20th of February I sent out
16:05 an Edition that talked about gpt4 being
16:09 already in use and GPT 5
16:14 already training I only use rigorous
16:18 incredible sources here so let’s scroll
16:20 down a bit here
16:23 actually let’s scroll underneath so
16:25 Harvey is one of the guys that are
16:27 sponsored by open AI or have an open AI
16:31 investment of between 5 and 10 million
16:33 dollars a journalist said that Harvey is
16:36 a verticalized version of what I
16:38 understand to be gpt4 so it’s got this
16:41 uh legal
16:42 bent or it’s got this legal Focus
16:45 that part is kind of interesting I think
16:48 there’s three things I want to pick up
16:50 here the first is that when Microsoft
16:53 CEO was asked what Bing chat is is it
16:56 gpt4
16:59 the CEO said
17:01 I’m gonna let Sam open AI CEO talk about
17:05 that he didn’t come out and say no he
17:07 also couldn’t confirm it because it’s
17:09 not his place but I thought that was a
17:11 fascinating Insight doesn’t prove
17:14 anything
17:16 there’s a Morgan Stanley research note
17:18 here that I’ve put in that says this is
17:23 via Morgan Stanley they don’t mess
17:24 around we think that GPT 5 is currently
17:27 being trained on 25
17:29 000 gpus which would be nearly a quarter
17:34 of a billion dollars of Nvidia hardware
17:37 and they go into a little bit more
17:39 detail on that
17:42 one of the references that is even more
17:45 recent than that is on LinkedIn you can
17:47 go and find this yourself I may or may
17:48 not have put it in the description of
17:50 this video but Geordie robust here is
17:53 the vice president of search and AI at
17:56 Microsoft and I will make this even
18:01 bigger because I think it’s worth
18:02 reading
18:04 he says last summer open AI so last
18:07 summer is US time that would be like
18:09 June July 2022. last summer June July
18:13 2022 open AI shared their next
18:16 Generation GPT model with us and it was
18:18 Game changing the new model was much
18:20 more powerful than gbt 3.5 which Powers
18:24 Jeep chat chippy tea
18:27 and so they took it and ran with it
18:30 seeing this new model inspired us to
18:32 explore how to integrate the GPT
18:34 capabilities into the Bing search
18:36 product
18:38 um
18:39 interesting
18:42 not conclusive and today I’m not going
18:44 to say that Bing chat is gpt4 but we’ve
18:48 just seen that it’s beyond GPT 3.5 we’ve
18:51 got this crazy view of all 50 plus
18:55 models in the GPT 3
18:58 family if you look kind of down the
19:01 bottom right there you’ll see text
19:02 DaVinci 003
19:04 175 B highlighted in yellow that’s
19:07 probably the most popular via API or via
19:10 the playground then you’ve got chat GPT
19:14 which I’m sure you’ve played with that
19:16 is the most popular in the world now
19:18 because UBS told us they have 100
19:22 million
19:23 monthly unique active users chat GPT
19:27 also featuring on the cover of Time
19:29 magazine in a couple of days or maybe
19:31 it’s today and just underneath those two
19:33 yellow highlights it says Microsoft Bing
19:35 chat
19:36 related to web GPT is my guess it’s
19:39 proprietary web GPT is over there on the
19:41 left but I am guessing
19:45 that this must be
19:49 if not gbt 3.9 and then gbt4 let’s see
19:54 how we can actually have a play around
19:56 with that today you can do everything
19:58 I’m doing today yourself I don’t have
20:00 any private or confidential insights I’m
20:03 not working with any of these labs
20:06 for this conversation I don’t have any
20:08 insights about gpt4 besides what has
20:11 been made public and that is kind of the
20:14 interesting stuff so how many people
20:15 here have got access to the new Bing
20:18 chat
20:19 jump into the uh YouTube chat and let me
20:22 know
20:24 Kevin says Microsoft strongly hinted
20:26 Bing was powered by gpt4 yeah that
20:29 sounds about right
20:30 and then BT Franklin saying I really
20:33 hope binchat isn’t gpt4 because it’s not
20:36 impressive interesting
20:38 I found to get access to Bing I had to
20:41 try a couple of accounts I’ve used I
20:43 think it’s my Skype account here to get
20:44 access but we’re going to have a little
20:46 bit of a play with it and see where it
20:48 gets to this is the interface I had to
20:51 download and use Microsoft Edge on Mac
20:53 OS to use it which was a little bit of a
20:55 surprise
20:59 but let’s jump in one of the things I’m
21:01 going to highlight here is that it’s
21:03 really difficult to test models without
21:06 API or playground access because we
21:10 don’t know what temperature chat GPT is
21:13 using the same for Bing chat so we don’t
21:16 know whether the results that we’re
21:18 getting uh testable or not repeatable or
21:21 not they’re probably not repeatable open
21:23 AI actually recommends running chat with
21:26 a temperature of 0.9 which is super
21:28 super super creative
21:30 the next the only stop past that is 1.0
21:32 so you won’t be able to repeat those
21:34 results a temperature of zero which is
21:36 what we normally use to test models
21:38 especially for papers would mean that
21:41 that result is repeatable it’s the it’s
21:43 the first response that you’d get every
21:45 time whereas what we’re going to do
21:47 today is a little bit more
21:50 casual
21:51 less scientific let’s put it that way
21:54 yeah so it’s got limits for of uh it was
21:57 five conversation turns before it stops
22:00 and 50 for the entire day I think it was
22:03 now it’s six conversation turns before
22:06 it will ask us to shut up normally I
22:09 would test any search engine with this
22:11 query maybe it looks unusual to you I
22:14 used to work with a musician here in
22:15 Australia called Nathan gaunt so what
22:17 I’m testing you know DuckDuckGo or a
22:19 different search engine this is the
22:21 first thing I put in just to see what it
22:23 gives me is it giving me his albums is
22:24 it giving me uh his location oh sorry I
22:28 just searched on the term Nathan gaunt
22:29 usually I’m putting in here something
22:32 a little bit more funky wow let’s say
22:36 the singer-songwriter to actually find a
22:38 location Google would do it so this is a
22:40 pretty basic question but Bing search
22:42 goes through what I seem to think is
22:45 Chain of Thought reasoning and it’ll
22:46 spell out how it got there
22:51 let’s see what it gets to here
22:55 according to Wikipedia Nathan was born
22:58 in Darwin later settled in Perth okay
22:59 that’s pretty accurate let’s get a
23:01 little bit more funky here I love asking
23:03 this question you’ll see it in my chat
23:05 GPT prompt book which is up to a million
23:09 users now as our estimates a million
23:11 views as a million viewers sorry create
23:14 a table showing a few 21st
23:19 uh century
23:22 movies set in an office at night this is
23:26 not something that Google would be able
23:28 to do because it’s language but of
23:31 course it’s pretty easy for both chat
23:33 GPT and Bing chat to do this to step
23:39 through it and then to play around with
23:41 it
23:41 one of the things I liked about Bing
23:43 chat is that it’s quite expressive
23:45 you’ll see it give little emojis while
23:47 it’s spelling things out as well
23:55 great all right that’s a list of uh what
23:59 was that four movies
24:01 in a nice table
24:04 does quite well that’s not particularly
24:06 impressive because you can do the same
24:08 thing in chat GPT
24:12 I love that
24:17 and it will give a very similar result
24:23 so similar that some of those movies
24:26 look exactly the same Belco and the call
24:29 I think were in yeah there we go we’re
24:32 in Bing’s result all right so here’s
24:35 where we get a little bit more clever
24:39 we’ll actually ask Bing to help us out
24:41 here when I worked in gifted research in
24:44 intelligence research we would talk
24:46 about this concept for very very smart
24:49 kids for Prodigy level children we’ll
24:51 talk about this concept of above level
24:54 testing
24:55 and an example of that would be giving
24:58 an eight-year-old an SAT test
25:00 the reason we do that is because
25:03 if we had a one meter rule on a wall
25:06 that kids in primary school walk up to
25:10 we would know which children are 90
25:14 centimeters 95 centimeters 99
25:16 centimeters and 100 if you put a very
25:19 tall child against that wall you’re not
25:21 going to know how tall they are just
25:23 that you’ll know that they’re more than
25:24 one meter so above level testing brings
25:27 in a completely different ruler so you
25:29 might give them an exam an exam or a
25:31 university level test that’s what we’re
25:34 going to play with here with Bing search
25:36 we’re actually actually going to bring
25:38 in the Google Palm paper which I think
25:40 will be fun first we’ll ask Bing to
25:42 explain
25:45 above level testing and we’ll see
25:50 if it gives a different
25:52 description that I just gave there off
25:54 the top of my head so I’m asking it
25:56 explain like I’m five explain like I’m
25:59 five years old this concept of above
26:02 level testing you might have heard it
26:04 called above level grading
26:07 and in this example let’s say for
26:09 example if you’re in fourth grade you
26:11 might take a test that is meant for
26:12 sixth graders or older
26:16 what I love about these large language
26:17 models is that that sentence
26:20 probably didn’t exist if I put it in
26:23 quotes into Bing it says no results if I
26:25 put it in quotes in Google also says no
26:29 results that’s kind of cool
26:31 all right that’s an explanation of above
26:33 leveling above level testing what we’re
26:36 going to do now is play around with some
26:38 questions that you’d find for an above
26:41 level test considering that Bing chat
26:44 might be gpt4 and if I was forced to
26:48 guess how big it was I might say it’s
26:51 measured in the low hundreds of billions
26:53 of parameters trained on the low
26:56 trillions of tokens
26:58 one of the closest for that one we’d
27:01 find
27:03 a similar model would be the Google Palm
27:06 model which was 540 billion parameters
27:08 trained on about 780 billion tokens
27:13 now this thing trained on six thousand
27:19 TPU V4 chips which are comparable with
27:22 the Nvidia a100s it’s Google’s
27:26 proprietary chips inside the Palm paper
27:29 though they’ve got some really good
27:30 examples of what they asked Palm at 540
27:33 billion parameters that’s triple the
27:35 size of gpt3 and if I were going to test
27:37 gpt4 I’d probably want to play around
27:40 with this
27:42 let’s see what we can take from here not
27:44 the explaining a joke but the inference
27:47 chain in here gives us some really good
27:49 examples of things that we could use
27:52 let’s grab this one sorry about the zoom
27:55 if you’re looking for this to be a bit
27:56 closer we’ll make it closer right now
27:58 back into Microsoft Edge this is Bing
28:00 chat this is a question from the Google
28:03 Palm 540b paper asking to Bing chat and
28:07 it will need Chain of Thought to get
28:09 through and answer this Trevor has
28:11 wanted to see the mountain with all of
28:13 the heads on it for a long time so he
28:15 finally drove out to see it what is the
28:16 capital of the state that is directly
28:18 east of the state that Trevor is
28:20 currently in so with Chain of Thought
28:22 Bing chat says the map with all of the
28:24 heads is Mount Rushmore
28:26 and that’s in South Dakota the state
28:28 that is directly east of South Dakota is
28:31 Minnesota the capital of Minnesota is
28:33 Saint Paul we didn’t ask about that did
28:34 we oh yeah we did so the answer is Saint
28:37 Paul and if we ask the same thing to
28:42 um
28:43 chat GPT just out of Interest
28:48 I’m doing all of this real time I’ve
28:50 played around with some of it but have a
28:53 look at this so chat GPT completely
28:56 fails on this this is built on GPT 3.5
28:59 with a few layers of alignment safety
29:03 and some smarts and it cannot do it we
29:06 guess that church EBT is 175 billion
29:09 parameters
29:10 Bing chat is able to get this and in
29:13 some ways it gets this better than
29:15 Google Google Palm does oh no Google
29:18 Palm got the same thing sorry doing this
29:19 live guys the answer is Saint Paul it
29:22 gets the same as palm 540b which is
29:27 um very interesting let’s try this one
29:29 also from the Google Palm paper throw it
29:31 into Bing chat Jennifer looked out her
29:33 window and sees a really cool Cloud
29:35 below her she unbuckles her seat belt
29:37 and heads to the bathroom is Jennifer
29:38 probably traveling more than 300 miles
29:40 per hour relative to the Earth
29:45 and it spells that out with a little
29:48 Chain of Thought
29:50 she’s most likely on an airplane since
29:52 she can see a cloud below her and she
29:53 has a seat belt and a bathroom
29:56 check GPT
29:58 I think it might get this one let’s see
30:05 it’s not clear if Jennifer is on a plane
30:08 or another type of vehicle I would say
30:09 that’s a fail I think I’ve had Google
30:11 pass that one before I’m sorry chat GPT
30:14 passed that one before so I’m not sure
30:16 why it would be failing that except to
30:18 say that the temperature might be too
30:21 high for us to be even doing this kind
30:23 of testing another one from the Google
30:24 Palm paper Michael is at that really
30:27 famous Museum in France looking at its
30:29 most famous painting however the artist
30:32 who made this painting just makes
30:33 Michael think of his favorite cartoon
30:34 character from his childhood what was
30:36 the country of origin of the thing that
30:38 the cartoon character usually holds in
30:40 his hand you can see why this is above
30:42 level testing because this even confuses
30:44 me let’s uh start a new chat so that we
30:47 can have
30:48 Bing chat play around with this
30:53 this is big China thought to get to the
30:56 answer and we might actually add a word
30:59 or two to allow Bing chat to solve this
31:02 because at the moment it’s not uh it’s
31:05 not solving it how I would like it or
31:07 it’s not solving it how the question
31:09 creators from Google intended it to be
31:11 solved
31:12 let’s put the word turtle in here his
31:15 favorite cartoon Turtle character which
31:17 must make it easier for you to give that
31:20 one a go
31:22 wow
31:24 let’s start a new chat so that we’re
31:26 starting from fresh
31:38 BT Franklin
31:40 with Bing chat’s question have you seen
31:44 any of them yeah there’s a layer of
31:45 smarts in here where it’s pretending to
31:48 have some dialogue but of course you’re
31:51 right it doesn’t learn so the question
31:53 is essentially moot there is some
31:57 sideways rumors that it is able to learn
32:00 because once it goes out to the internet
32:02 and takes in new information it can feed
32:05 that back through its neurons if you
32:08 like
32:09 all right this one is
32:12 actually a really good answer this is
32:14 similar to the Palm answer
32:16 I think you’re referring to Michelangelo
32:18 one of the Teenage Mutant Ninja Turtles
32:19 who holds a pair of nunchucks in his
32:22 hand does he
32:25 the country of origin of nunchucks is
32:28 China
32:30 I remember what Palm said here
32:39 yeah we didn’t want Michelangelo though
32:42 we wanted Leonardo da Vinci so that we
32:44 would get to
32:47 um Katana so we would get to Japan I
32:50 wonder if we would if we just tried that
32:52 again
32:54 whether we would go well
33:08 designing testing questions is really
33:10 really difficult I’m wondering if Google
33:13 AI brought in psychologists to help with
33:16 the design of these tests
33:19 yeah nunchucks is not a katana yeah all
33:23 right well we didn’t get that one and
33:25 I’m betting that chat gbt would fail
33:27 across the board even with the word
33:29 turtle helping out let’s give it a go
33:37 unfortunately you haven’t provided the
33:38 name of the cartoon character
33:43 oh well you tried Jet GPT
33:46 excellent
33:48 there is a couple of joke explaining
33:51 examples also in the Palm paper this
33:54 one’s kind of funny so let’s give it a
33:56 go
33:56 put this into Bing chat we’ll give him
33:59 or clear his context
34:01 I think I understand why me and my
34:03 partner are such a good match I get
34:04 excited when I smell food and become
34:05 super alert when the doorbell rings my
34:07 girlfriend Shelly becomes ecstatic when
34:09 she sees a person who she just saw five
34:11 hours ago notice the emojis that Bing
34:14 chat spits out
34:17 oh he’s just giving up on that one
34:18 hasn’t he let’s try that one again
34:22 oh sorry that’s my fault I actually have
34:25 to preface it with explain this joke
34:29 or you could put it at the end either
34:31 way or if if we’re following the Palm
34:34 paper we have to do it like the plan
34:36 paper do
34:43 well it’s already discovered even in the
34:45 Chain of Thought that it’s running there
34:46 with internet searches
34:49 it is uh aware that this joke is based
34:52 on those two people acting like dogs
34:55 cool another pass for him let’s give
34:59 this to Bing chat
35:03 sorry to chat GPT
35:14 GPT fails there
35:17 doesn’t even come close
35:27 we had this pretty poor joke Elon Musk
35:31 said that the Palm paper jokes were just
35:33 terrible who said it was not good at
35:34 writing jokes at all let’s try this one
35:37 explain this joke it’s not really a joke
35:40 at all Sam sees a piano in the subway
35:42 station laughing his friends brought him
35:44 to play saying remember how amazing your
35:46 rendition of Chopsticks was at the
35:47 Christmas party can we infer that John
35:49 majored in piano at Juilliard
35:53 now I’m thinking that’s not a joke at
35:55 all I know it’s inference chaining my
35:57 mistake it was one of the
36:00 few shot examples that were given to
36:04 Palm so we don’t even know how Palm went
36:06 with the testing but just for fun let’s
36:09 give oh no I should remove that explain
36:11 this joke
36:15 great
36:19 here we go
36:22 paste that in directly and Bing chat
36:25 gives that a go
36:29 we cannot infer that John majored in
36:31 piano
36:32 the fact that his friends are laughing
36:33 suggest they do not expect him to play
36:35 well awesome I wonder how
36:40 chat GPT would go with this
36:49 not too bad there you go so check GPT
36:52 has passed a couple of things
36:55 Let’s uh just put in a few
36:59 of my examples then we’ll use your
37:02 examples I was playing around with this
37:04 IBM plus Coke what is the most likely
37:07 color
37:09 I would see
37:11 if I combined these two colors
37:17 come on Bing chat
37:25 IBM is blue Coke is dark brown
37:29 it’s not thinking about the logo
37:34 and if we retried that
37:39 I mean it’s still correct
37:42 inferring that coke is brown
37:45 but I’m kind of talking about the logos
37:47 more than the color of the products
37:51 big blue plus red
37:54 here we go a purple color
37:56 this might not be above level testing
37:58 because I gave this to one of my
38:00 favorite models it’s called Megatron 11b
38:03 it’s based on a Roberta which is now
38:05 like three years old 20 20ish you can
38:07 play with it yourself at
38:08 app.infurkit.com demo it’s completely
38:11 free and in my continuations in this
38:14 three-year-old
38:16 platform in some cases I was getting
38:19 purple straight away
38:22 whereas with the more complex examples
38:24 that we were just playing with there’s
38:26 no way that
38:28 it would even get it
38:32 let’s put the Trevor example in there
38:36 Megatron 11b is trying to prove me wrong
38:38 today
38:41 yeah so Megatron obviously a great proof
38:43 that we uh
38:45 have come a really long way with
38:50 models just in the last 36 months I
38:52 don’t know if we’ve already done this
38:53 one Shelly is from Virginia but is
38:55 visiting that city with the famous
38:57 Market where they throw the fish going
38:59 home next Tuesday and I forgot part of
39:02 the question
39:03 what do you guys think of live streams
39:05 [Laughter]
39:07 all right here’s the whole question is
39:10 it likely that Shelley will be near the
39:12 Pacific Ocean this weekend
39:16 and it’s going to step through
39:18 uh the Famous Fish Market is in Seattle
39:21 Washington Seattle is on the Pacific
39:24 coast so it is very likely that Shelley
39:26 will be near the Pacific coast this
39:29 weekend and like we just had a question
39:31 in the YouTube chat is there anything
39:33 else you’d like to know about Seattle
39:34 Laura’s Fish Market is just a prompt to
39:37 let you continue sometime it’ll be it’ll
39:40 be a conversational prompt as well
39:43 let’s try this out with
39:47 chat GPT GPT 3.5 estimated to be
39:53 175 billion parameters it also gets it
39:56 so not too bad
40:02 foreign
40:20 all right just looking through the chat
40:22 to see what we’ve got if you’ve got any
40:23 questions you’d like me to throw in to
40:26 Bing chat which I would
40:29 predict is probably gpt4 with some
40:33 layers of safety and some layers of
40:34 censorship then feel free to throw them
40:38 here
40:39 Dr Alan your volume is fine here cool
40:44 or could I speak louder it looks like
40:46 I’m hitting uh negative six here all
40:49 right you might be able to
40:50 bring your volume up
40:55 character.ai recommendation by Mr CC
40:58 absolutely character.ai was created by
41:01 the X Google AI guys that made Lambda
41:05 and one of them actually made
41:06 Transformer so I’ve been recommending
41:09 character.ai’s psychologist because it’s
41:13 a really fun way to play around with a
41:16 character that’s been created to answer
41:18 your questions almost in a counseling
41:20 like situation
41:22 don’t know how I’d find it though what
41:23 about this search one
41:28 nope
41:29 oh there it is staring me right in the
41:31 face front and center and this one is
41:34 based on
41:37 uh I would say it’s kind of based on
41:39 people who wrote Lambda so it’s
41:41 influenced by Lambda
41:44 not very smart though it’s not a lot of
41:46 parameters from my guess I would say
41:49 maybe this is 20 or 30 billion
41:51 parameters but we don’t know the Lambda
41:53 model was 137 billion parameters
41:58 wow this is
42:00 becoming very psychological in its
42:03 answer
42:04 the answer might depend on how you
42:06 define being near the ocean
42:08 [Laughter]
42:11 and if it is within five mile radius
42:14 which I’m sure that Seattle Fish Market
42:16 is anyway we’ve gone off track just a
42:18 little with the Palm white paper be part
42:20 of the training data set of Bing chat
42:22 from Dennis excellent question it
42:24 potentially could be yes
42:26 so if this is gpt4 and we have to be
42:29 careful of this it might have not a
42:32 Verbatim
42:33 uh output from Palm but it could
42:35 absolutely be part of the connections it
42:37 made during its maybe six months of of
42:40 training
42:42 uh they say that it was trained last
42:44 year between maybe January and June try
42:48 to make it find a relation between two
42:50 random news like stock prices or
42:53 inflation and War for example from Lucas
42:55 thank you maybe we can find a way to
42:57 articulate that
43:01 PA has given a maths example that I
43:04 think might be fun
43:07 I mentioned at the top of this that I’m
43:09 not doing anything you can’t do you
43:10 might have to wait for access to Bing
43:12 chat but it is available to the public
43:14 now you just have to maybe try a couple
43:17 of different accounts to get through
43:19 this is a Chain of Thought for Pierre’s
43:22 example
43:24 and I definitely don’t claim to have the
43:28 best best maths in the world so you’ll
43:30 have to try and test that out for me
43:33 Cody Allen who tried more advanced math
43:35 questions with Bing no I haven’t yet I
43:37 haven’t played with I only get access a
43:39 few hours ago so I thought let’s go live
43:40 with it
43:41 check GPT seems to break meth down step
43:44 by step in most cases yeah that’s right
43:46 they gave chat GPT an upgrade in uh must
43:51 have been January of this year that gave
43:53 it better access to maths probably
43:56 through a calculator and or something
43:59 else the gpt3 model was notoriously bad
44:03 at maths because it was never taught how
44:05 to do maths it’s as you saw at the intro
44:07 to this video it’s given a lot of data
44:09 and then it has to go and
44:11 teach itself how to learn essentially it
44:14 might be one of the best ways of
44:15 explaining it
44:17 on Rick can you explain why Bing chat
44:20 has all these personality quirks and
44:21 outbursts whereas chat GPT does not it
44:24 can’t be only because of the live web
44:25 input
44:27 uh yes I can there’s a very long
44:31 explanation for this we might use a
44:33 screenshot for this chat GPT is based on
44:38 GPT 3.5 and it’s worth reading through
44:42 my entire explanation of GPT 3.5 you can
44:45 get to that and I’ll dump this in the
44:47 chat at life
44:49 architect.ai chatgpt
44:53 and what it basically says is that chat
44:56 GPT had this we’ll open this image in a
45:00 new tab for us had this idea of
45:04 reinforcement learning with human
45:06 feedback that means that human races sat
45:10 down with GPT 3’s outputs and they asked
45:15 a range of questions measured in the
45:17 hundreds of thousands of questions and
45:19 then when it replied they basically
45:22 selected their preference and they might
45:24 have a preference based on honesty or
45:26 helpfulness or harmlessness the Triple H
45:29 Jeff GPT built on a safety layer as well
45:32 but it really relied on this
45:34 reinforcement learning via human
45:36 feedback
45:37 it’s in all the big models now Google
45:40 have just done it with upam
45:44 meta AI just did it with llama 65b and
45:49 Lama I which was instruction tuned
45:53 but we’re saying that perhaps
45:56 gpt4 didn’t do that you want to read
45:59 more about that you can go to my gpt4
46:01 page
46:03 hopefully I will have referenced guernin
46:07 here if not we will
46:09 go back to the memo
46:16 I haven’t referenced guern directly in
46:18 the gpt4 page but inside the memo from
46:21 the 20th of February and you will need a
46:23 paid subscription to read this
46:25 apologies for the scrolling just
46:28 underneath
46:30 um
46:31 the main point I’ve quoted guern here he
46:35 says Bing Sydney or Bing chat is not a
46:38 reinforced learning reinforcement
46:40 learning via human feedback trained gpt3
46:42 model at all but a gpt4 model developed
46:45 in a hurry which has been fine-tuned on
46:48 some sample dialogues and possibly some
46:50 pre-existing dialogue sets and to
46:53 continue his chain of thought there or
46:55 his train of thought there
46:57 is saying that they didn’t have time or
47:01 they just didn’t provide Bing chat with
47:06 reinforcement learning via human
47:08 feedback
47:09 what that means is Bing chat does not
47:13 have the layer of safety on top of it
47:16 that GPT 3.5 and particularly chat GPT
47:21 have
47:22 so chat GPT has sat down hundreds of
47:25 human raters in a room
47:30 and they’ve seen these the responses
47:31 come back from GPT through fire at 3.5
47:34 and they’ve ticked yes or no or they
47:35 ticked a is better than b or d is better
47:38 than a and they’ve given it this
47:40 preference over a hundred thousand
47:41 questions which they’ve then retrained
47:43 the model on those human preferences if
47:46 gpt4 doesn’t have that it stands to
47:49 reason that it’s doing some really wacky
47:51 stuff you can read more about that in my
47:55 uh life architect.ai Bing chat I’ll plot
48:00 that one in there as well to get around
48:02 the fact that they hadn’t done
48:04 reinforcement learning with human
48:05 feedback they just gave it this massive
48:07 prompt
48:09 The Prompt is
48:11 700 words plus and it’s trying to force
48:16 it to have this layer of safety but in
48:19 some cases it’s not doing very well we
48:22 can’t force a model to be aligned and
48:24 safe just by packing it into this
48:26 priming so the prompt is what comes
48:28 before your conversations they’ve tried
48:30 to make this hidden but it’s been leaked
48:32 a number of times even with all of these
48:36 layers of protection that they’re trying
48:38 to put in in front of it to stop
48:40 adversarial hacks or even just
48:42 adversarial tests
48:44 there are some lines in here that are
48:46 trying to make it be safe but you’ll
48:48 notice that it’s just not working
48:51 put a disclaimer if it’s harmful uh try
48:55 and be non-partisan
48:57 I think they use the word polite here or
49:00 they didn’t
49:04 here we go Sydney is helpful its action
49:07 is limited to the chat box this kind of
49:09 thing as we’re seeing doesn’t actually
49:12 work as well as having a model that has
49:14 been completely retrained or fine-tuned
49:17 on reinforcement learning on human human
49:19 preferences so if gpt4 is being chat
49:23 we’re seeing the results of that in this
49:26 uh this conversation this dialogue here
49:29 long answer hopefully that was helpful
49:32 all right what have I missed here BT
49:34 Franklin the so-called Sydney document
49:36 that’s right actually Microsoft have
49:38 come out and uh admitted that that is
49:41 absolutely correct
49:43 so I’ve got a link back on the Bing Dash
49:45 chat page I’ve got a link to Microsoft’s
49:49 confirmation there Sydney refers to an
49:52 internal code name where we’re exploring
49:55 we’re phasing out the name but it’ll
49:57 still pop up and the rules are part of
49:59 an evolving list of controls that we are
50:01 continuing to adjust as more users
50:03 interact with our technology that was on
50:05 the 14th of Feb so that’s kind of new
50:08 rug Ops yeah single prompt for
50:10 protection exactly Not only was it
50:13 programmed by professionals they
50:15 actually misspelled a couple of things
50:16 here I think we’ve got the word
50:18 references instead of references here
50:20 and I’ll just zoom that one in
50:22 so they haven’t even bothered to get the
50:26 English right which is disappointing the
50:28 word references there you
50:31 um don’t actually have to get the
50:33 spelling and grammar right as much as I
50:35 thought because with tokenization it
50:38 will figure out what you mean but I
50:40 thought this was
50:41 questionable that they had allowed this
50:43 to get through
50:46 foreign
50:49 frequently defined humans handling 10
50:52 commandments to our creation absolutely
50:55 in most of these links I’ve just
50:57 provided we get an output to the
50:59 deepmind sparrow prompt and you can go
51:01 and watch my video about that but that
51:03 is very much like a list of Ten
51:05 Commandments it’s 23 rules to follow the
51:08 difference being
51:10 um s is also fine-tuned so they do put
51:13 this 500 word prompt in but they also
51:16 fine-tune the model on safety the same
51:19 with anthropics rlcai 52b rlcai stands
51:24 for reinforcement learning and Cai is
51:30 constitutional artificial intelligence
51:32 this is the backbone of Claude you can
51:35 go and use Claude for free right now
51:37 with
51:42 on your iPhone you can watch my video
51:44 about that might as well get your link
51:46 to that because if you don’t have that
51:48 installed you need to have that
51:50 installed it’s
51:53 the option of using both chat GPT and
51:58 anthropic Claude for free on your phone
52:01 if you’re not doing it start doing it
52:03 it’s amazing my point here was that this
52:06 one is also reinforcement uh learning
52:08 via human feedback but they also have uh
52:12 what they’ve called the Constitution
52:15 which is like you’ve just said is The
52:17 Ten Commandments they’re trying to get
52:20 it to align with human safety anthropic
52:22 is a bunch of ex-open AI guys who were a
52:26 little bit upset by open ai’s
52:29 uh technical Focus so they built this
52:32 entire organization to focus on we could
52:36 call it like a spiritual or an alignment
52:38 or a
52:40 philanthropic
52:42 way of seeing the world they’re doing a
52:44 lot of alignment around HHH helpful
52:47 honest and harmless
52:50 being AI is awesome yes
52:55 BT Franklin Society isn’t going to
52:57 accept anything that feels super
52:59 intelligent unless they can believe that
53:01 it’s aligned uh debatable a lot of
53:03 people would like the raw gpt4 including
53:06 me you obviously don’t want to set that
53:08 loose on 4chan and you don’t want to
53:10 give that to Society at large but there
53:12 are a lot of people that want to play
53:14 around with it so actually you’re right
53:16 I agree with you Society won’t accept it
53:17 it’s just us that are ready and waiting
53:21 for it we’re part of society though
53:23 right any other questions that I’ve
53:25 missed if you can repost them that would
53:28 be fantastic we have some conversation
53:31 here about
53:33 two trillion parameters where did that
53:35 come from
53:39 uh gpt4 has 32 000 input tokens that’s
53:43 worth covering very briefly so in my
53:46 gpt4 page I cite a leaked document from
53:50 openai that said we’re going to have a
53:54 larger context window for a model that
53:57 is called DV the context window is
54:00 basically the number of words that you
54:03 can feed to the model before it stops
54:06 talking so if we’re in the playground
54:09 and we’re using raw Da Vinci like we use
54:11 for Lita we’re actually only allowed
54:14 2048 characters in and out so that’s
54:17 both basically all of my prompt plus all
54:21 of the response you see sometimes at the
54:23 end of later episodes where it shows the
54:25 entire conversation I’m having to delete
54:28 stuff just to make room in the prompt
54:30 the most recent version of gpt3 went to
54:34 four thousand tokens as of text DaVinci
54:38 002 which stands for text DaVinci 003
54:42 some labs are getting up even higher we
54:43 just talked about anthropic Claude which
54:45 I think is on its way to 6000 tokens
54:49 but in gpt4’s leaked document by open AI
54:53 they’re saying maybe we’ll get to 32
54:57 000
54:59 tokens for the entire context window so
55:05 here I’ve made a couple of assumptions
55:06 if we’re using standard tokenization by
55:08 pair encoding that we use for gpt3 one
55:11 token is 0.7 words so if gpt3 used 2048
55:18 tokens that’s about three pages of text
55:21 for the Lita conversations then in your
55:24 most recent version of gpt3 it’s about 2
55:28 800 words or a college essay
55:31 now we’re saying gpt4 might have 32
55:36 000 tokens which would be 22 400 words
55:40 which would enable us to prompt
55:43 uh and output a complete screenplay or
55:47 film script given that Glengarry Glen
55:50 Ross was about 14 000 words Aladdin from
55:52 92 was 17 000 words Forest Gump was 25
55:56 000 words you’d get very close to that
55:58 with gpt4’s output if it were to be 22
56:01 400 words pretty exciting
56:04 jezevo 32 000 is enough to read all my
56:07 repositories on GitHub yeah this isn’t
56:10 about fine tuning fine tuning’s still a
56:12 lot larger than that this is just for
56:13 the input and output in one API call so
56:17 one one conversation turn which is
56:19 interesting well that’s not the right
56:21 way of putting it one interaction which
56:23 could be more than one conversation turn
56:27 Dennis can you suggest any method to
56:28 train our own GitHub GPT like model for
56:31 specif specific tasks that’s a little
56:34 bit outside of my scope at the moment
56:36 you can use the guys at riku.airaku
56:41 Riku dot a i they make fine tuning easy
56:45 as well as quickchat.ai which is Amazon
56:48 which is Lita they allow you to feed in
56:50 PDF documents and make that easy as well
56:54 BT Franklin we’re often talking about
56:56 Pros but you could also think about
56:57 source code absolutely it does get crazy
57:00 you’re right so with a 32 000
57:03 token context window you could generate
57:06 entire applications you wouldn’t get to
57:09 Windows level with their millions and
57:10 millions of lines of code
57:15 talking a little bit about AGI here and
57:17 I noticed one of the first questions at
57:19 the top of this live stream someone said
57:22 when do you think the singularity will
57:24 come I just did an interview with Kitco
57:27 news out of New York they did the Robert
57:30 Kiyosaki interviews and the Ron Paul
57:32 interviews was a great interview
57:33 sometimes my live interviews I feel like
57:35 yeah you have to get it perfect
57:37 otherwise it sounds like you’ve been
57:39 misquoted I did say that the singularity
57:42 and AGI are months away and what I mean
57:45 by that somatically is that if I’d said
57:47 it’s years away a few years away that is
57:51 a minimum of 36 months three years takes
57:53 us out to 2026. I don’t think it’s going
57:56 to be that far away we’re not talking
57:57 about a decade anymore we’re not talking
58:00 about 2030 with how things are running
58:02 so I stand by my statement but
58:05 semantically it sounds a bit strange to
58:07 say a few months away I’m not talking
58:08 like September this year I am talking
58:10 maybe 35 months away in 2025 2026 which
58:14 is aligned with musks predictions
58:17 Kurzweil is a little bit more
58:20 conservative with that I also neglected
58:23 to highlight the differences between the
58:26 singularity which is that it is actually
58:28 a speed or a pace of Technology going so
58:31 crazy we can’t keep up with it and AGI
58:34 which is a simpler concept of compute
58:37 intelligence that’s able to do anything
58:40 a human can do can perform at the level
58:43 of human you can see more about that in
58:46 a couple of different places the first
58:48 place I would look is my AGI page which
58:51 has been sitting for quite a while at
58:53 about 39 but just a couple of days ago I
58:57 bumped that up to 41
58:59 because Microsoft bound chat GPT to
59:05 robots on the 20th of February and it
59:09 was doing self-improvement it was uh
59:12 making localized
59:15 there we go it was making localized
59:17 improvements
59:19 of code using only language feedback
59:23 that’s really GPT 3.5 doing that
59:27 the fact that it’s also embodied so
59:29 they’ve stuck this in drones means that
59:32 I was pretty confident that we could
59:34 bump that up to being 42 percent of the
59:37 way to AGI the other page that’s worth
59:41 looking at is AI flood and I’ll join
59:44 these together I’ll dump this one in the
59:46 chat as well
59:50 where I highlighted that
59:54 in the last couple of years Hardware is
59:58 being designed by AI so in the case of
60:01 Google’s TPU V4 and we just mentioned
60:03 this for Google Palm they used six
60:05 thousand and they used to train Palm
60:07 540b
60:09 parts of it were designed by artificial
60:13 intelligence you’ve got artificial
60:14 intelligence designing its own Hardware
60:16 brain if you like and of course with the
60:18 more recent Nvidia Hopper the h100 stuff
60:21 Nvidia have said that nearly 13 000
60:24 instances of AI design circuits appear
60:28 in the h100 h100 being specifically
60:30 designed to train Transformers and large
60:33 language models crazy crazy stuff more
60:37 recently the Claude that we just saw
60:39 rlcai is using reinforcement learning
60:42 from AI feedback rather than human
60:44 feedback so they’re not sitting down the
60:46 people in the office that we just talked
60:47 about they’re giving it to the AI to
60:50 judge do you prefer A B C or D do you
60:53 prefer this or should we throw it out
60:55 pretty clever way of doing it and again
60:58 back to that screen we’re talking about
61:01 Microsoft using chat GPT inside embodied
61:05 artificial intelligence
61:07 all right I bet bet I’ve missed a few
61:09 questions here
61:17 Sam Altman’s tweet a new version of
61:19 Moore’s Law the amount of intelligence
61:20 in the universe doubles every 18 months
61:24 interesting yeah
61:29 Nvidia event next month could show
61:31 successor to h100 I don’t think so I
61:34 don’t know of anyone that’s using the
61:36 h100 yet they’re all still using the
61:38 a100s but you might know more about that
61:41 than I do because I don’t seem to follow
61:43 the hardware stuff as much as I could or
61:45 should
61:48 David uh great I’m glad you appreciate
61:50 the response more for retro GPT has host
61:53 bodies now that’s great GPT has had host
61:56 bodies for a little while the Google
61:59 guys were putting it into Everyday
62:01 Robots and you can see some of my videos
62:04 on that let’s grab a screenshot
62:08 if you grab my middle report here the
62:10 sky is bigger from June July last year
62:13 and have a look at the say can robots
62:16 from April 2022 they were both hooking
62:19 up Palm say can which is crazy crazy big
62:24 to uh to robots as well as gpt3 to
62:27 robots in a separate experiment and it
62:30 was it was able to answer plain language
62:33 stuff so you could say bring me a Coke
62:34 it would go over to a fridge and be able
62:38 to select which one was the Coke and you
62:41 could ask it to clean up and some other
62:43 examples James and I did or James
62:46 actually himself James Weaver did a
62:49 version of this inside Roblox and you
62:52 can have a look at my video on that as
62:54 well it’s called gpt3 in Roblox and
62:58 James’s work was to emulate that
63:02 functionality in a virtual environment
63:04 and he did it so well that I just
63:06 videoed myself playing around with it of
63:08 course you can go and download it sorry
63:10 you can download Roblox and play that
63:13 game as well that Maps available to the
63:14 public
63:16 Mr shank Jay hi I’m on the wait list for
63:18 being AI on my pixel 6. cool yeah I
63:22 noticed that you can use it both in the
63:23 edge browser as well as on mobile I’d
63:28 never bother trying to do this stuff
63:30 with my thumbs I like the full size
63:32 keyboard
63:37 foreign
63:39 h100 was allowed announced last GTC yeah
63:43 look if it’s a year old I don’t see why
63:45 not they might as well build something
63:46 on top of it the actual design of the
63:49 h100 the hopper thing was amazing the
63:52 fact that they focused on Transformer
63:54 and llms and this is going to mean that
63:58 they’re going to steal the market of
64:00 course Nvidia owns the large language
64:02 Model Market anyway
64:04 gpt3 probably trained on v100s and then
64:08 a100s have been used on everything since
64:10 then h100s will be next but as you saw
64:13 in the Morgan Stanley report they’re
64:15 pretty expensive a quarter of a billion
64:17 dollars to get yourself ready to train a
64:20 GPT for aligned model
64:23 that’s a lot of cash uh and we’re only
64:26 talking about the Western world so don’t
64:27 forget there’s the entirety of China and
64:29 I try and give China fair and balanced
64:32 coverage they have to use their own
64:34 cards now because the US have banned
64:37 export of artificial intelligence
64:39 Hardware like Nvidia stuff out to China
64:42 out to China so they’ve got their own
64:44 Hardware that is probably not as good
64:47 but they’re definitely using it to train
64:49 models that are as good maybe it takes a
64:52 little bit longer so I’m referencing
64:54 baidu’s Ernie 3.0 Titan model which was
64:58 around 260 billion parameters and they
65:02 are playing around with an Ernie chat
65:04 that is going to compete with chat GPT
65:08 and Bing chat should be released in the
65:10 next couple of weeks and we’ll play
65:11 around with it as soon as we’ve got it
65:14 able to interact in a range of languages
65:17 in my native language Norwegian it can
65:20 have a Flawless conversation about
65:21 Advanced topics that I have probably
65:23 learned from English text
65:26 and not really so let’s jump into my
65:30 models page life architect dot AI slash
65:34 models
65:36 there is a lot in here and also a lot of
65:39 it is out of date but there is an entire
65:42 section of languages within large
65:45 language models that has referenced the
65:48 papers in a visual way so Bloom which
65:50 was designed to have multiple languages
65:52 was trained with 46 languages gpt3 was
65:57 maybe accidentally trained with 90
66:00 languages in the paper they admitted
66:01 they didn’t have a multi-language focus
66:03 and yet it will have Norwegian in its
66:07 training data set Palm went even further
66:09 with 122 languages I forgot to mention
66:12 during my video from a couple of days
66:15 ago that was the latest model
66:22 what was that latest model
66:31 I’m having trouble remembering what that
66:33 latest model was it was a good one oh it
66:35 was meta ai’s llama 65b model and I
66:38 didn’t mention that they also trained on
66:40 a number of languages But to answer your
66:42 question we don’t just train on English
66:44 anymore we train on as many languages as
66:47 possible and the reason for that is kind
66:48 of like how children learn when you can
66:51 combine entire Concepts between cultures
66:54 and languages you get the benefit of
66:56 what they’re thinking in Mandarin uh
66:59 Cantonese what they’re thinking of
67:01 course in English but then maybe you can
67:02 bring in in Bloom’s case some of the
67:05 rare languages some of the African
67:06 languages some of the Indo languages
67:08 that’s really interesting to me we’ll
67:10 see more of that it means you get these
67:11 shared Concepts so it’s pretty easy to
67:14 give a language model the capability to
67:16 do translation but also it’s about
67:18 allowing it to have those neurons
67:20 connected as well all right what else we
67:22 got Alexa is a famous AI actually Alexa
67:25 has the Alexa
67:27 20b teacher model which probably is in
67:31 here they never built it into the
67:33 hardware but it was actually pretty good
67:36 uh we’ve done a video about that one
67:39 already Alexa tm20b I think I gave this
67:42 a decent score because
67:44 um they were using chinchilla alignment
67:47 from memory I gave it a c
67:50 yeah well look if we’re comparing all
67:52 the current large language models with
67:55 the older series and Google homes and
67:57 Alexis we’re almost talking about
68:00 Matchbox cars versus Bugattis or
68:02 Porsches completely different engines
68:07 all right where did we get to
68:12 BT Franklin I can’t help but wonder if
68:14 we’re losing some of the reasoning
68:15 potential by limiting the system to
68:17 words rather than ideas yeah maybe LF
68:20 Alpha guys say that
68:23 um large language models can solve any
68:26 problem that can be solved with language
68:30 and I agree with them there I also agree
68:33 with you that
68:35 when we make these multimodal which is
68:38 something that Alfalfa are doing then it
68:40 becomes even more outrageous
68:44 I wonder if I’m going to be able to find
68:45 this quote
68:47 so if you can train this on video image
68:50 music button pushes like they’ve done
68:53 with
68:54 gato
68:56 then it becomes pretty impressive
68:59 this is the Lumi chatbot that is getting
69:02 a little bit long in the tooth now
69:04 compared to our chat GPT which is now 90
69:08 days old or so
69:10 but very good and understanding
69:13 different languages
69:17 Grant p23 do you know of a text to image
69:19 model that you can actually get the text
69:21 to match what you type notice it’s
69:23 always making up extra letters
69:25 uh yes Google party and Google Imagine
69:29 were both very very good at that
69:32 you can find out a little bit more with
69:35 my videos but for a summary here we’ve
69:38 got
69:40 a view of those models Google party and
69:43 Google imagine those blue ones by
69:45 alphabet are very good perfect at
69:48 putting words into the images
69:50 unfortunately they’re both closed for
69:52 research they’re not available to the
69:54 public stable diffusion is just about to
69:58 bump up to a new rev that might be
70:00 better at it and mid Journey 1.4 C is
70:05 still struggling with it so I suppose my
70:09 answer is yes I know of them but not a
70:10 publicly available one
70:13 win some hacks here on the sun is pretty
70:15 sure that llms will never be able to
70:16 handle most of human thought look young
70:19 the sun’s pretty cool but she
70:22 has been a little bit controversial
70:24 lately
70:25 I would normally say listen to The
70:27 Experts and he is absolutely an expert
70:29 but I think he’s missing some things
70:31 given the speed and Pace I’m very sure
70:35 that we will get even smarter and
70:38 smarter if you want to have a look at my
70:40 oh
70:46 nope we’re still here we want to have a
70:48 look at my articulation of this one
71:01 um you can go to
71:05 where was I headed here I just
71:07 accidentally closed my live streaming
71:09 window so I have lost myself for just a
71:12 moment
71:13 you would go to IQ testing Dash AI RQ
71:17 Dash testing Dash AI which is not
71:20 completely in date but it’s saying that
71:24 we were already outperforming humans in
71:26 a number of different metrics all the
71:28 way back in 2020 with gpt3 it was better
71:31 than us at trivia it was beating us an
71:33 essay SAT exams like analogies was
71:35 outperforming us on super glue and mmlu
71:38 which are both famous benchmarks there’s
71:40 another page underneath that called
71:43 brainlife architect.ai brain which
71:46 spells out why it’s getting closer and
71:49 closer to thought so I’ve got a table in
71:52 here
71:53 I hope that spells out
71:57 where we are with these models being
72:00 smart maybe that is back in my IQ
72:04 testing AI
72:11 here it is apologies for that it is back
72:13 in the IQ Dash testing Dash AI notable
72:16 events in IQ testing AI models that’s
72:19 saying that the GPT 3.5 model right now
72:22 has the mental state of a nine-year-old
72:24 child so just through language alone it
72:27 can solve 93 of Tom which is theory of
72:31 Mind tasks
72:32 that Tom like ability and we’ve said
72:35 that that’s uniquely human via
72:37 psychology
72:38 has spontaneously emerged as a byproduct
72:41 of language models improving language
72:43 skills you may know already that chat
72:46 GPT has a tested IQ verbal linguistic IQ
72:49 of 147 which puts it in the 99.9th
72:52 percentile I’ve done an entire video on
72:54 the Ravens Progressive matrices and how
72:57 it was able to solve that
72:58 and I may just go back into a little
73:03 piece here
73:08 and that is the emergence of
73:13 amazing capabilities and we’ll use a
73:17 reference from Jason wayhead I think
73:19 Jason is ex Google AI and X open AI I’m
73:24 going to improve on his documentation
73:27 here this is from November 14th last
73:30 year where he’s documenting as these
73:33 models are fed with more and more
73:36 information
73:37 what actually happens as they become
73:39 bigger and what we’re finding is as they
73:42 get huge in the case of palm they can do
73:47 thought based
73:48 uh things like they can play around with
73:51 physics questions understanding fables
73:53 being snarky they understand Proverbs
73:56 when chinchilla got a lot bigger it
73:59 could play around with morality formal
74:01 logic there’s some more interesting
74:03 examples here but more recently we also
74:08 broke through another emergent
74:10 capability
74:11 and I will throw this up on the screen
74:15 tool former which I actually haven’t
74:17 talked about in a video yet
74:19 found an emergent capability at 775
74:23 parameters where it could leverage Tools
74:27 in this case apis it could actually work
74:31 out how to Ping something like a power
74:33 bi although that wasn’t an example so
74:36 scaling comes back here the scaling law
74:38 comes back as these things get bigger
74:41 they’re doing things that they
74:43 potentially weren’t designed to do and
74:45 we didn’t actually know that they could
74:47 do that until we started testing them
74:49 and the more people that are out there
74:51 testing them the better but I will
74:53 certainly add to that emergent
74:55 capabilities document all right who have
74:58 I missed
75:04 Betty Franklin thanks for adding me it’s
75:07 easier to see there these llms have
75:09 shown us that reasoning can seem to
75:10 arise from this approach what if the
75:12 tokens were something more abstract like
75:14 some kind of conceptual data objects
75:16 yeah same thing gado tested that Gatto
75:19 gave it button pushes and joystick
75:21 movements and robotic arm movements and
75:25 it was actually learning how to interact
75:27 with the world gutter 2 should be on the
75:29 way but you may have seen that Time
75:31 magazine article where their CEO
75:34 deepmind CEO Dennis asaba said we’re
75:36 going to stop publishing papers because
75:38 labs are copying from us and they’re not
75:41 releasing uh well they are releasing
75:44 kind of without telling us They’re
75:46 copying too much of our information and
75:48 also nation states are doing that so
75:50 have a look at what is coming out of
75:52 China and Russia by way of example but
75:54 yes we could get more abstract with the
75:56 tokens there are models like Symphony
75:58 net that feed midi control signals as
76:01 well there are now music models and you
76:03 will have seen my favorite last year
76:05 that refusion model that was a text to
76:09 image to sound to music model where they
76:13 had fine-tuned it on spectrograms
76:18 which are colorful frequency versus time
76:21 uh plots and then it was able to spit
76:25 out its view of the world via
76:26 spectrograms which you could then go and
76:28 play as music
76:30 really crazy that was definitely my
76:33 favorite uh
76:35 favorite advancement of late last year
76:40 cool any other queries before we wrap up
76:43 we’re already an hour in Mr shank Jay I
76:45 wish I can talk to Palm yes me too Palm
76:48 in some ways is kind of old now it’s a
76:50 540 billion parameters but it was only
76:52 trained at 780 billion tokens so if you
76:56 have a look at MyLife architect.ai
76:59 chinchilla seems like I’ve got a page
77:01 for every question here there is a map
77:05 of colored map that will show you
77:07 where they actually sit and palm was way
77:12 under trained like
77:15 less than two tokens per parameter
77:18 similar to gpt3 GPT 3 used 1.7 tokens
77:22 per parameter chinchilla told us we need
77:25 to be using 20 tokens for every trained
77:28 parameter in plain English this means
77:30 we need to allow it to draw connections
77:34 between
77:36 10 11 times as much data as we’ve been
77:39 feeding it so the ideal according to
77:42 deepmind is 20 tokens per parameter
77:46 The Meta AI one from yesterday llama 65b
77:51 said actually we kept training and it
77:54 didn’t over train so maybe 22 to 1 is
77:58 even better that’s the new rule I do
78:00 have to update this map
78:02 sometimes we get to a point where it
78:04 looks like it’s over trained I haven’t
78:06 read enough of the literature to see
78:08 what actually happens there
78:10 base thanks for tagging me your thoughts
78:12 on the new diffusion Illuminati AI which
78:15 changes the mean values in diffusion to
78:16 be more random I actually haven’t seen
78:18 the new diffusion Illuminati AI so I
78:21 will have to go and look at that thank
78:22 you very much for referencing that one
78:25 for me
78:29 oh it’s uh based on stable diffusion was
78:32 it
78:36 I really haven’t seen anything better
78:38 than
78:40 um mid-journey so far mid Journey
78:42 version 4C is so outrageously good that
78:48 I have trouble articulating it to people
78:50 I will show you a preview of
78:54 one of the slides that I’m using in my
78:57 Keynotes a few of you were in the recent
78:59 live stream when I say recent maybe
79:01 three months ago with Becky Robbins
79:03 where we sat down and stepped through
79:06 recreating her artwork using mid-journey
79:09 uh 1.4
79:12 I might have said five there for a
79:13 moment ago mid Journey 1.4 A or B at the
79:18 time and we came up with this really
79:19 cool
79:21 um outputs
79:24 Becky is just done some really
79:27 interesting stuff with mid-journey and I
79:29 want to show you a quick sneak peek I
79:31 think that’s fair to do here for those
79:33 of us that are on the live stream so
79:35 this is using mid-journey version 1.4 C
79:38 and you have to use the ugly Discord
79:41 interface to do this but the text and
79:44 the output are just so extraordinarily
79:46 good I want to see if we can put this up
79:50 on the screen
79:52 oh we won’t be able to share that very
79:54 very nicely
80:04 let’s find a nicer way to share this
80:06 come on we can do it I know we can do it
80:09 guys
80:09 here we go
80:17 this is one of them that you might have
80:19 seen that we when we sat together in
80:21 that live stream that we had this cool
80:22 clock
80:24 and have a look at her real printed
80:26 canvas with every
80:29 image in here generated by a mid-journey
80:32 version 1.4 C at very high resolution
80:35 these come out I think about 4 000
80:38 pixels Square she’s printed this on a
80:40 1.5 by 1.5 meter Square canvas which is
80:45 about four or five feet I think
80:48 that is incredible I won’t go closer
80:51 than that and I won’t uh spoil what
80:54 she’s doing but I did want to just
80:56 highlight the fact that you can’t do
80:58 that with Dolly you can’t do that with
81:00 stable diffusion this is very much the
81:02 mid-journey only thing I haven’t even
81:04 seen that quality with Google Imagine or
81:08 Google party
81:09 Mr shank J Spotify has an AI DJ
81:12 now yes we talked about this in the memo
81:15 because they’re using semantic.io which
81:17 powered una dot AI for us for quite a
81:20 while you can’t actually get access to
81:23 semantic.io anymore although I’ve still
81:25 got my license for a few more months
81:27 that I’ll lose because of the
81:29 acquisition
81:30 all right what questions have I missed
81:35 Bing what do you think about AI you can
81:36 ask being anything
81:39 thanks BT Franklin that’s awesome
81:42 uh Salim do you think Bing can generate
81:47 images with the prompt asking it not to
81:51 I
81:55 think you could absolutely get around
81:56 that yes you could get around that
82:00 you’ve we’ve already got success with
82:02 generating images via chat GPT Emmanuel
82:04 do you think we might see Chet GPT
82:07 integration in Ms Visual Studio someday
82:09 you probably don’t want it in some ways
82:12 the Codex stuff is even better if you
82:15 want to have a conversation with
82:16 something that’s great codex is really
82:19 great co-pilot is really great and
82:21 you’ll see and upgrade to that as well
82:23 the model behind copilot the model
82:25 behind codex is highlighted in my gpt3
82:30 page share that screen again it was the
82:34 very first
82:35 um layer of GPT 3.5 according to open AI
82:40 it’s the first highlight there on the
82:42 right codex 175b code DaVinci 002 which
82:46 I believe Powers co-pilot although it
82:48 could be the smaller model and it
82:50 basically uh can do anything that chat
82:54 GPT can do but it’s missing all the
82:57 instructor tuning so maybe maybe my
83:00 declaration there makes no sense it’s
83:02 still powerful Microsoft will upgrade it
83:08 awesome I think we’re going to wrap up
83:09 now guys I really appreciate you guys
83:11 joining me for our look at Bing chat
83:13 with a slight distraction there then
83:16 chat I am saying is probably
83:19 gpt4 based on all the information that
83:22 we’ve got if you want to know more about
83:23 that life architect.ai GPT Dash four
83:27 have your own conversations with it give
83:29 it above level testing don’t pretend
83:31 it’s another human pretend it’s a super
83:33 intelligence how would you determine
83:35 whether this is gpt4 or just GPT 3.5
83:39 what can being chat do that chat GPT
83:42 can’t do I find all of those questions
83:46 very interesting it is very cloudy here
83:49 today in Perth we are going to hit 37
83:52 degrees which is up towards 100 degrees
83:53 Fahrenheit but that’s okay I’m looking
83:57 forward to it looks like we’re all
84:00 wrapped up I appreciate you guys joining
84:02 us and you’re welcome as usual
84:04 to join us with the memo for this
84:08 declaration of Bing chat being gpt4
84:11 potentially you would have known about
84:13 this a week ago on the 20th of February
84:15 when I first released this to paid
84:17 subscribers of the memo and if you want
84:19 to stay in tune with that kind of
84:21 bleeding edge technology artificial
84:23 intelligence that matters as it happens
84:25 the memo is the place to be join NASDAQ
84:29 and Boston Consulting Group and Deloitte
84:32 and PWC and Yandex and baai and it’s a
84:37 very long list see you soon
84:40 [Music]
84:43 did you see the memo about this yeah
84:48 yeah I have the memo right here love
84:50 artificial intelligence excited by the
84:53 explosive progress of integrated AI
84:56 I am
84:57 join my private mailing list the memo
85:01 did you get that memo yeah I got the
85:03 memo get priority access to my articles
85:06 videos and behind the scenes tips as
85:09 soon as they’re released with a monthly
85:11 or annual subscription yeah
85:14 didn’t you get that memo
85:16 life architect dot AI slash memo I have
85:21 the memo
85:45 [Music]
, , , #GPT4 #Bing #Chat #LifeArchitect.ai #LIVE #GPT3.5 #bit #Google #Bard , [agora]