NeurIPS – Rumours from the trade show floor

Insights HPC


Generally, trade shows follow the sun and tourists to popular vacation destinations. Everyone loves a conference in San Diego or Orlando! The recently rebranded NeurIPS (formally NIPS) took a different road this year and visited Montreal in early December. Montreal is one of my favourite cities but in early December it’s the season for cold, cloudy weather and infrequent freezing rain. Here's a quick rundown on my experiences at the conference.


Thinking back, maybe it was a stroke of genius to host NeurIPS in a chilly location - hardly anyone skipped the conference sessions to go outside! Even the normally exuberant McGill students were focused on their finals and looking somewhat pensive. With NeurIPS in town, Montreal took the opportunity bestowed on it to showcase the tremendous amount of grassroots AI development taking place and the interesting, entrepreneurial start-up’s utilising deep neural network technologies which are sprouting up from every corner of the city. It was an excellent, thought-provoking few days in Quebec.

So onto the conference itself - NeurIPS was the thirty-second 'Conference on Neural Information Processing Systems' – the technical heart of AI. I felt rather undressed with my degree in Electrical and Electronic Engineering! Like other AI events the conference has been growing exponentially and boasted over 8,000 international attendees this year. Perhaps 90% were academics both students and staff. The enterprise attendees were either research boffins, compute equipment vendors or surprisingly recruiters trying to lock-in the next generation of hot data scientists and machine learning engineers.

This is the only AI event I've been to that I’ve seen more than one quant hedge fund attend. Quant funds are usually solitary, secretive creatures so to have fourteen or more recruiting next to one another was a bit of a shock.

Garnering admission to the conference is a trick, all the conference passes sold out within the first eleven minutes! I managed with an exhibit hall pass which was ideal considering my differential equation math etc. is rather rusty.

There is often a dichotomy between the compute resources needed for academic versus enterprise AI. Many of the papers presented at NeurIPS focused on the mathematical techniques which can help DNN training converge on a robust solution. Consequently, validating the results perhaps a hundred times is enough. Often this is done on a workstation or high-powered GPU augmented server. Enterprise AI often requires training DNNs on millions of images of all types, in all lighting conditions to produce a robust DNN fit for a production product. This often requires a large HPC compute cluster with hundred or thousands of CPU and GPUs.

I spent a fun day visiting all the booths in the exhibit hall trying multiple ways to convince the recruiters staffing them to introduce me to their IT management. The jury is considering carefully whether I succeeded.

Most attendees were very interested in the sessions and less so in the exhibits but during the breaks and evening social the exhibits were mobbed. The booths revealed some interesting and orthogonal things about their owners. Bosch has created an internal AI team which acts as a consulting group to guide point development projects and refresh their next generation of automotive technology – very smart.

The financial service companies are driving AI into a multitude of areas from program trading algorithm development, to data-mined news feeds to complex risk assessments. However, I was truly surprised that about 10% of the attendees were from Google or Deep Mind. The other hyperscalers were also well represented but not to this extent. Tom Diethe, from Amazon’s retail branch’s AI team in Cambridge is applying AI to sticky problems in supply chain management and really making a difference.

After the exhibits I worked the show app to facilitate interesting one-on-one meetings during the breaks which resulted in some fascinating discussions. Sizigi Studios are applying DNN technology to automate the coloring of animated video they have the potential to accelerate and repatriate much of the animation manual post design activity from east Asia to the USA. Perhaps a better, more entrepreneurial approach to making America great again, than simply trade tariffs!

Lightmatter, a Boston USA company, are developing a photonics CPUs to accelerate low-power consumption computation. I had a great chat with two of their physicists learning what aspects of DNN training compute could be synthesized in optics and perhaps more importantly what could not. It appears that we are still some way from implementing non-linear functions in optics but overall, I was impressed with what feels like a modern-day reincarnation of analog computing at light speed.


I also encountered my first machine vision application which likely doesn’t need retraining once in production. This small custom application uses a phone to read the final electric meter reading before a smart meter is installed and does QA on the manual final reading done by the technician. The machine vision is currently 97% accurate and is more than enough to identify process and equipment failures leading to large scale last reading errors. Once retrained the DNN would likely only improve slightly and the company elected to focus on their next puzzle.

All in all my three days at NeurIPS was well worth the trouble and I was yearning to stay longer and avoid the red-eye to London for far less technical discussions. I can’t wait until NeurIPS 2019 in Vancouver.

Following our first AI and HPC Field Trip to Iceland in October, we're pleased to announce we will be running a second Field Trip early in 2019. If you're working within the field of intensive AI and deep neural network development, and would like to be considered for attendance, please get in touch - bob.fletcher@verneglobal.com


Written by Bob Fletcher

See Bob Fletcher's blog

Bob, a veteran of the telecommunications and technology industries, is Verne Global's VP of Strategy. He has a keen interest in HPC and the continuing evolution of AI and deep neural networks (DNN). He's based in Boston, Massachusetts.

Related blogs

DevOps. Stacking up to be a Common Theme at ISC18

Yesterday was a good day to be in Frankfurt. All of the majors in the supercomputing universe descended upon the Messe Frankfurt to begin ISC18 with a series of training seminars. For my morning session, I chose Getting Started with Containers on HPC through Singularity put on by the team at Sylabs, Inc. I have been tracking the progress on Singularity in the HPC community since before Sylabs was founded by CEO Greg Kurtzer in an effort to bring the technology of root secure containers into the realm of enterprise supported software. I was excited to hear about the progress that Sylabs has made and to see where the future of containers lies for the broader HPC community. If I was forced to sum the tutorial into a single portmanteau, it would be DevOps. After this session, it is clear to me that the world of DevOps that has been created in the cloud native universe is on a collision course with HPC. And the future of science says that it can’t happen soon enough.

Read more


TrueHPC at Industrial Scale

As we prepare to descend upon Frankfurt this week for ISC18, science will take its appropriate place at center stage. Throughout the week, the gathering will hear how through science we are challenging the world’s toughest problems, dissecting those problems down to their foundations and then building them back up by methodically moving every minute detail that has been learned into the powerful realm of the supercomputer. Science shows us that the best innovations are created by literally starting from scratch and building from there.

Read more


hpcDIRECT scales new heights with SATAVIA

As I take a short break from the exhibition floor here at another excellent ISC18 in sunny Frankfurt, I’m delighted to announce SATAVIA as the latest customer to join our bare metal high performance computing (HPC) platform – hpcDIRECT.

Read more

We use cookies to ensure we give you the best experience on our website, to analyse our website traffic, and to understand where our visitors are coming from. By browsing our website, you consent to our use of cookies and other tracking technologies. Read our Privacy Policy for more information.