Using Machine Learning to write the Queen’s Christmas Message

In their excellent book “The Indisputable Existence of Santa Claus” Hannah Fry (aka @FryRsquared) and Thomas Oléron Evans (aka @Mathistopheles) talked about using Markov Chains to generate the Queen’s Christmas message. You can read a bit about that here. After reading that chapter I asked Hannah and Thomas if they had considered repeating this using recurrant neural networks. A couple of years ago Andrej Karpathy wrote a blog that he summarised as follows:

We’ll train RNNs to generate text character by character and ponder the question “how is that even possible?”

In his blog he posed the question:

It looks like we can learn to spell English words. But how about if there is more structure and style in the data?

and went on to train an rNN to write text in the style of  Shakespeare. I recommend you read his blog to see the examples and get an idea of what is going on.

You might also want to watch this video from the excellent Computerphile on a similar subject.

In a previous blog I decide to have a go at using rNNs to generate lyrics for my favourite band New Model Army. Thomas was kind enough to send me the text for the Queen’s previous speeches that he and Hannah had used in their work, and I thought I’d see how good rNNs were at generating a Royal Christmas message.  I didn’t do anything particular fancy here…just followed the code and examples in Karpathy’s original blog on a fairly standard home desktop machine.

Here are some of the results… (I take no responsibility for any text generated by the neural network :)).

One parameter that can be adjusted to generate the output is ‘temperature’. This takes a value between 0 and 1. Lower temperature will cause the model to make more likely, but also more boring and conservative predictions. Higher temperatures cause the model to take more chances and increase diversity of results, but at a cost of more mistakes. Running with a temperaure of 0.5 we get:

i want to send a special message of the human spirit are much to be for everyone this is a happy and part of the christmas story so in the same streed. we are all truly new zealand in an example of the commonwealth has an equal place in the lives of men and women and what they have to show that they are all life in their own peoples of the commonwealth which we have come to the sacrifice of all the fortunes of hope and there are many heart of our grandchildren the presents the prince of wales and will have a preacher group of the commonwealth. on a strong and spreading the responsibility to the commonwealth who are much on the courage of the modern world and the commonwealth games and there are many others at home and the rest of the world have been inspired by the courage and enjoyment and the present peace of a commonwealth which is how to promote the games and from all walks of life and to become a growing peace on earth which we should me to see the strength to try to do so in the world today. it was a great conflict remain inspired by the courage and prince philip and i were able to see buckingham palace which the most special year for the commonwealth is to be movement and in a different furmestand and they are all source of spirit all over the world have shared a moment in the year and the christmas message is to be consed rather a quicter and the contribution of the family of nations and i have meen and wonderful in both these ordarisasions or a common sense and encouragement. this year i am speaking to you from the same family make the world in the world. the christmas message is so many of them and i have difference to the best of ourselves. and this year there are something to look at the world they are may be a life of all the world is the commonwealth is a sense of belonging to the world i was well as they are all sort of moment the more depression

I’m not sure this makes a lot of sense, and there are a few oddities/errors in the text.

I tried lowering the temperature to 0.3:

i hope that the christmas message is to be there and it is not only a very happy christmas to you all. i am speaking to you from the commonwealth and i have been able to see that the commonwealth and i have been deeply touched this year i am speaking to you from the same family friends and communities as the future of the commonwealth and as i have always given to many of you will have heard of the more conscious of the commonwealth and around us we should be able to see that the commonwealth and i have been deeply touched the strength to try to do the same family gathering in the commonwealth and i have all been about the future as well as a great comfort to us all the problems of the commonwealth and i have seen in their own way the problems of the commonwealth and i have been deeply touched the world the progress of the commonwealth and around us we are common throughout the commonwealth who are struck by the state of the commonwealth and i have depended to see in the world today that we should remember those who have seen in their own way a celebration of the child who was born at christmas time for families and friends will never be very different problems but it is not only a time for reflection and confidence to the commonwealth and i have all been about the world that we should not be our lives and to remind us of the future. i am speaking to you from the commonwealth and i have been deeply touched the world that we can all try to make a splendid birthday and the commonwealth and i have been floods and sadness and the best of the world have been able to discuss the best of ourselves. i believe that this christmas day i want to send a special message of hope in the face of hardship is nothing new that of the commonwealth who have seen in their lives in the commonwealth and in the commonwealth and i have been able to discuss the best of ourselves. we are all live together as a great daily and its war the commonwealth

As suggested the the result here is a bit more predictable/boring.

It is also possible to prime the model with some starting text using. This starts out the RNN with some hardcoded characters to warm it up with some context before it starts generating text.

Using a temperature of 0.5 and prime text of ‘Each Christmas’ we get:

each christmas is a time for reflection and grandchildren the more depends of a dedication of the commonwealth and around us of the future of the commonwealth games in the rich and proud of life in a great religions and its members. but i am also speaking by instince in the right through the members of my family and i have been great and personal responsibility to our lives and they say to me that there are many happy or so many countries of the future with the prince of peace and happiness and self-respect for the commonwealth and is a happy and prosperous future of the world we will be achieved with the opportunity to help them are a contribution to the powerful ways of spirit and learning to see the problems of the commonwealth and i have seen in their inestivation the peoples of the commonwealth and i have been for the better but that is possible to the people and the christians it’s all those who are great practical form we have all been for them to be a chance to get the gift of a carren of different lives the service of their people throughout the world. we are all live in the rolin to our children and grandchildren the present generation to see this country and arrived the same for one another which is most popularity and the rest of the world and all the great commonwealth. pronomed news the responsibilities for the duty of events which we have all been the holidic of science. it is they all who are so there and shared heritage and sometimes in saint paul’s and much considerate and to communication the human spirit. and they can be a contralle commonwealth and a firm landmark in the course of the many servicemen and women who are broken for many that can be a precious given us to be a witness this continuing that we can all be the contribution of the commonwealth and as we all have all the features of the love of hope and goodwill. this year has constantly will be overcome. i believe that this year i am speaking to you from the hope of a determination and continues to all

Using a temperature of 0.5 and prime text of ‘This  past year’ we get:

this past year has been one to see that what i have been for the commonwealth and around us we are thinking of the problems show in the face of the fact that he wanted the strongly that we should show many of them service is the birth of the commonwealth. it is a contribution of a lives of family and friends for parents and religious difficulties at the commonwealth and as we all share the future as the future of the future. it was a moment and that the christmas message of the things that have been god shown in the earliest care and for the commonwealth can give the united kingdom and across the commonwealth. it is a helping one of the march of people who are so easy to go but the rest of the world have shaped for their determination and courage of what is right the life of life is word if we can do the state of the same time last month i was welcomed as you and that the opportunity to honour the most important of the thread which have provided a strong and family. even the commonwealth is a common bond that the old games and in the courage which the generations of the commonwealth and i have those to succeed without problems which have their course all that the world has been complete strangers which could have our response. i believe that the triditings of reconciliation but there is nothing in and happy or acts of the commonwealth and around us by science the right of all the members of the world have been difficult and the benefits of dreads and happiness and service to the commonwealth of the future. i wish you all a very happy christmas to you all.

So there you have it. Not sure we’ll be replacing the Queen with an AI anytime soon. Merry Christmas!

Advertisement

Using Recurrent Neural Networks to Hallucinate New Model Army Lyrics

I decided to follow the example of Andrei Karpathy to see if I could use recurrent neural networks to hallucinate New Model Army lyrics. I decided to train a 3-layer RNN with 512 hidden nodes on each layer. Full technical details are here. If you want to compare the output with the original lyrics you can by reading them here. Here is the output, not bad considering the relatively small size of the input (if you’re not a fan you’ll have to take my word for it):

the thick black days below where the desert with their backs the mountains in the blood red sun the sky i can hear us in the darkness they got their victorious and the black sky and the secrets come back to the sky the sky i see you and i can hear us all breathing i can hear us all breathing i can hear us all breathing i wanted to see him face and the lovers and the backstreets and the country breaks the shadows of the foghorn on the last breath there is still sprays on the ridge and the hands are on the breaking skies the boys are heading good for the father and the lights go out the sea and i was so let them on the earth and the days of the end we all feel the lights are blazing all the screen when i was watching the fires and the birds of brave still we are the falling sky i was a believe in the darkness trying to the statue calling of the father and a cars the power in the sky i have seen the streets head into the shadows of the burning sand and the way the falling of the storm in the wind of the streets of the winding wheel to the back the pressure moves the will we started away into the wall with the city where we are gonna fight with a blood of passion and you will be found of your face is still across the sky the trees i am drinking i am asking what is we wanted before the stars there is a bown star of the winding shots and the heart and the sun stars and the stands protects the lights and the days are lands and i am happy to be forever but they have got to be innocents the world is bow to the wind blows where the sky we are still not worried now i will hold on to the world and i am sorry and we are gonna fight we will be hope that is comes and we were walking out in the road the rain and the hands are far the same thing one one by one of the cold there is a fire so much to the sky i have seen them so much to tell us what is is the way that is i see a little corner of the sky and the steel walls they think that is you can hear us all breathing i am  looking for you i think i was make us surprised i am not worried now i am not worried now i want no second who wants to see the corner of the far away from everything i have ever wanted when i am screaming i am screaming i am looking i am never going back there i am never going back to the seething sun the way that is change the road i do not know what is the coming skies and drums they live by the back i started at the bar and the grey i will be some place that is i can hear us all breathing i can hear us all breathing i can hear us all breathing i can hear us all breathing i can hear us all breathing i swear that is i have never been and the far horizon it could be stormclouds and it was we are still here the hours in the flag and the hands are frozen the stars below the doors we were out of sun on the corner and the things that is i do not really want it i can hear us all breathing i can hear us the passion is not  where i am warming i am sick of the conversations we are gonna die the way they should not believe it still you have been so well you want to be here the sound of the war we have done we are drifting and here in the world in the sun i was not a more than they can look up and i will be some own while the clocks and the great car is all come to see a man fould on the world we were born in the blood dream i can hear us all breathing i am swilling in the end we stand on the top of the water the shadows of the sun and the cheap coats of the cooling towers and the wind blows through the side of the call train we are gonna fight we watch the world while they wait for the crowds and the chemical walls that is we are not war and our hearts they fall the long gods take their children and the lost the steps from the path of the shadows of the sun on the road and all the shadows come crashing in and staring and freedom and the grey and the grass when we go out there is a passion past the blood sun and the falling of the gill of the fire and the back in the wind blows through the backstreets of the desert that is we are still behind us and i am part of us the things that is you were not of the discotheque and the faces call in the sky in the blazing sun and the sun from the sky i have  still the strung place that is you want to look down to the sea on the chest of the millions of the cold cold the dance glow their dead and the sun we are gonna fight we were found on the dead we are we are not a great laugh and laughing down the sky i swear that is we are gonna fight we are still the pains that is i have seen to think that is we have done with their faces are the same the passion in the sea and the right of the miles of the faces that is we have to be in the bar and the fead and the waiting lights the storm when we were strange to see a man fall or to see him fly the city the spell blew the cold charge there is words are like with the world and the rooms are still spark in the empty streets there are gone before we will be hard and gone when we said that is we can hear us all breathing i am looking for you i am  alone and the more and the earth start and the end it takes all of them will come the same our little shoulder sun and still the black and the lights go out the way they are gone feel the scars and the money i am sorry and the fire and the world is never good faces they are gone before and the noise with the hollows and the rain and the hard of the cardy blows when the hands are the same in something i am one of the blood where the sky i have something to come i am finding i am die and cheat of the days of the sun in the courtry of the water close my hunger the killing day and the grey and the streets are out and the breathing of the end the world is gone but the race is can go out here and i can feel the stars put out of the sky i am drifting out in our little time and the streets are lighting the desert their many counting down the streets are in our reason conversations to the faces come back there is a car and there is a purity that is we can feel my hands and i will meet you in just to spend the bridge and the black and the sea and the rain and the back and still we are born in the power in the frozen claim and i will be standing there i will come the stars there is a siren passion street with more than i can hear myself from the great braves shot and the spoilt generation and i am  back and while you will still the same thing that is i see them in a part of us but you are still here for the shadows of the water the wind blows through the trees and the shadows of the water the heart of the flags are the devil in the country the streets and the television promises staring out of the crowd it was just the world while they are so lost i am sorry there i have been so much to talk to the beautiful car and the truth is a frightened days to the push of the sky in the space of the ash is a curse the boys are dancing the stags are black dark and the open driving sun in the sky i saw the way that is i do not really want to have to call the desert in the country from the lights beneath the clouds of the construction of the foot scare and the fire and the angry gods we would be here and we will see them in the party is    so much to the way i was driving fast and the world and the bad our breathing sky the statues beneath the clouds of silence and the corner of the glamour breaking the same as the scands the blood and the grey south and the scream of the discotheque of the flashing days that is i cannot see what is i have seen more tonight i am sick of the power i am  not and nothing was worthy and i know i will be a sen for a city the world has the same and i will tell you what is i have been and the silence she says and i will see them we were born in the chemical trucks close the darkness and the great roads and the city the sun and the streets are billion stars and the seasons change and there is a children who think they should not do that is what is i am not warming the crackles of pain and we blood and still here all the drops at the world in the sky in the battle of the morning the shadows come the sea and the falling of the water wheels the truth is river on the world is die in the shadow of the streets of her hands are lands and the promises stars beneath the black the crackle and the mountains and the lights below the streets are laid the same and the hard storm are all the seasons the way that is you love me in the world i want the bottle of the passion in the empty power is the power and the sea and the hands are falling stars behind the road and the seasons change and there is a burnt of my heart behind the truth is the same on the streets are living and the back the fire but i can feel it all of us to go confasy the camera while they will  be home but you will embrace it and i am still that is you are not a power i was not the promises i saw the way that is i am never going back there i will be mountains and there is nothing   to do you can watch your eyes and i will never be stormclouds and the time but there is no warning all the things that is i can hear myself breathing i see the smoke and the lights are sard and the changes and the rain and the seasons change only the seasons change only the wind blows through the silence stars and the way the boys go when it is all vanity i feel like the time is standing here for the ones that is we are better than them with the sky in the moon and the sea and the man from the days are gone before the new world it is not  a west for the consuming flags and the gravesing streets and the call and the shoulders and the long land closer to the sky i am lost the fashion and the back and the sea as the prayer and the stains they are still the bridges are cracking on the sea and the bridges are not a black the world we will meet the changes we are still still here and we are gonna die i want