AVX-512: A Performance Disappointment for Deep Learning on Skylake-X October 2020 I have been hesitant to publish this analysis, as the findings were rather underwhelming. However, the technicalnuance makes it a story worth sharing. Recently, I acquired Skylake-X processors with AVX-512 support and optimizedthe DeepTrainer engine to take advantage of these wider vector instructions. Given […]

Dynamic Graph Architecture: Enabling Composite Networks DeepTrainer Technical Update The evolution of neural network architectures moves rapidly. Recently, the rise of Deep Residual Learning (ResNet) caught my attention. The core concept—introducing “skip connections” where the input of a layer is added to the output of a later layer—presented a challenge for DeepTrainer’s existing architecture. My […]

Optimization Log Sometimes performance gains come from weeks of rigorous profiling. Other times, they fall into your lap when you least expect them. In my recent work updating the DeepTrainer interface and cloud infrastructure, I stumbled upon significant speedups that pushed our benchmarks to new heights. The 64-bit Surprise Historically, I maintained the core project […]

I have added a new interface for the DeepTrainer library that makes the algorithms accessible through a web interface using queries and which makes it possible to deploy the library in the cloud. The service implements Open Authorization (OAuth) using bearer tokens. Here are the interface functions it implements at the moment: User account registration: […]

I have been reading about the ONNX file format recently that has been created internally by Google, but before I delve myself into protocol buffers I still needed an easily readable (read: debuggable) file format to exchange neural network states. I know that eventually I will have to end up supporting ONNX so I did […]

After I was done with the WinForms test harness update I noticed that the application was leaking memory like the Titanic after meeting an iceberg. I have managed to narrow the issue down to the OpenGL window – then I realized that the best course of action I could take here is to implement both […]

I did a complete overhaul for the algorithmic structure of the library because when I first wrote it I kept extending an existing structure as new ideas came without being concerned about code redundancy. So now I decided to get rid of all redundant code pieces in all my classes. I ended up with a […]