The LEGO® Group to Pilot Audio and Braille instructions for the Blind.

At the Recognised Fan Media days in Billund, earlier this year, I attended a presentation by Matthew Shifrin. Matthew is no ordinary LEGO Fan. He has been blind since birth, and has never seen a LEGO brick. He had a story to tell us, and he delivered it with passion.

A Labour of Love

For his 13th Birthday, a friend, Lilya gave Matthew a copy of 7573 Battle of Alumet, one of the sets tying in with Prince of Persia movie. At the same time, Lilya presented him with a large binder containing step by step instuctions – hand typed on a Perkins Brailler.

Lilya helped Matthew by providing instructions to further sets, refining the language standards used along the way. They posted the instructions on line, and received requests from the parents of blind children to provide instructions for this set or that. As a two person operation, with Lilya typing the instructions and Matthew testing them, it became impossible to keep up with the demand.

Over the years, Matthew and Lilya developed over 20 sets of instructions, from small sets through to some of the large Creator Expert sets, including the Sydney Opera House and Tower Bridge. Examples of these ‘translations’ can be found here. Before, some of these larger sets have instructions that are over 800 pages long.

“Blind people learn by touching things” Matthew said to us.”If, for example I wanted to fell what the Statue of Liberty felt like, I would try and climb it. The trouble is, that would get me arrested. So, if I built that same Statue of Liberty out of LEGO, then I am able to become intimately familiar with why its shaped the way that it is. And thus, learn more about the parts of the world which are untouchable to blind people.

“I realised that I needed to share this with other blind kids: they deserved it so we started out website – lego instructions for the blind.“

“Lilya died [in 2017] And I promised myself that I would not let this project rest until LEGO themselves took action. And now they have.”

Matthew took the project to the LEGO foundation, via the MIT Media Lab and the LEGO Creative Play Lab.

Olav Gjerlufsen, from the LEGO group , explained that they took the idea to the Austrian Research Institute for Artificial Intelligence, who set about developing LXFML – LEGO Exchange Format Mel script: essentially a translator that is able to take the visual instructions, and convert them into language: from there it is a relatively easy exercise to convert them to speech or Braille.

“Matthew’s story demonstrates the power of LEGO play. It brings people together, helps to build confidence and sparks creativity. It has been an honour to work with Matthew, his passion and energy are truly inspiring. But most importantly his project will help visually impaired children around the world experience the same joy of building and pride of creation that all our fans feel”, says Fenella Blaize Charity, Creative Director, LEGO Group.

• 

• 

• 

• 

Available in English as a free service for all through the accessible website www.legoaudioinstructions.com, the first four instructions to be launched include a set from LEGO® Classic, LEGO® CITY, LEGO® Friends and LEGO® Movie 2™. Consumers can either chose to hear audio instructions using their screen reader or with audio provided by the LEGO Group, or alternatively chose to read the instructions using a Braille reader. Depending on consumer feedback on the four pilot instructions, which will be collected until the end of 2019, the intention is to launch more Audio & Braille instructions first half of 2020.

“As I build a set I develop a better sense of what a building looks like and how it is laid out and constructed. For blind people LEGO sets act as miniature 3D substitutes for real-life buildings in lieu of two-dimensional photographs. LEGO bricks allow me to see things that are impossible to explore by touch, such as the arches of a Middle Eastern palace or the towers of the London Tower Bridge.

I would like to get my instructions out to the blind community. I would like every blind person to be able to download the instructions, buy a set, have a sighted person sort the pieces, and feel on par with a sighted builder. I want every blind person to feel that the once impossible is now possible; that he or she can now build a miniature LEGO world.”

Matthew Shifrin

Matthew appeared in a short documentary two years ago, discussing his passion for LEGO Bricks, and the initial development of the Instructions for the Blind. You can watch it right here:

Since his presentation to us at the Recognised Fan Media Days, he has also presented a talk at TEDx Suffolk University in June 2019. It is essentially an expanded version of the talk he gave us, and well worth your time. He has some great ideas about accessibility for not only building LEGO, but also comic books, as well as travelling.

While these instructions show radical innovation, there’s still a lot of progress to be made in terms of further developing the AI software and automating the process. The long term ambition is to add more languages and support all future product launches – most importantly however, is to ensure fun and high quality learning through play experiences. 

The pilot launches just four months after LEGO Braille Bricks were announced – an initiative aimed at supporting young children with vision impairment to learn Braille in a playful and inclusive way. Co-developed with the LEGO Group, both projects have been funded by the LEGO Foundation.

Personally, I think the idea of developing AI to convert the illustrated instructions into natural speech is pretty cool. I happen to have a couple of the sets involved in the pilot program waiting to be built, so I will present my experience in a few weeks time.

If you have enjoyed this article, why not share it with your friends and communities, and leave your comments below. Until next time,

Play well!