Resource-efficient Neural Networks

Seminar Type: - None -
Project Status: Open

While machine learning is traditionally a resource intensive task, embedded systems, autonomous navigation and the vision of the Internet-of-Things fuel the interest in resource efficient approaches. These approaches require a carefully chosen trade-off between performance and resource consumption in terms of computation and energy. The tremendous memory and computational complexity of e.g. Convolutional Neural Networks (CNNs) prevents the deployment on resource-constrained systems.

As a result, recent research focused on CNN optimization techniques, in particular quantization, which allows weights and activations of layers to be represented with just a few bits while achieving impressive prediction performance. However, aggressive quantization techniques still fail to achieve full-precision prediction performance on state-of-the-art CNN architectures on large-scale classification tasks.

In this work we aim to use and explore CNNs with reduced-precision weights and activations for image and speech classification problems. Furthermore, we are interested to explore them using alternative quantization approaches.

 We offer:

  • existing code of CNNs
  • benchmark data

 Your Tasks:

  • simulate CNNs in Python on the GPU using Tensorflow
  • analyze the implemented systems in terms of accuracy and performance
  • analyze the shortcommings of current CNNs - suggest alternative training methods
  • [contribute to scientific work in form of a paper]

Your Outcome:

  • learn to implement and simulate Neural Networks on a GPU
  • learn how to solve classification problems with CNNs
  • get a good background in applied machine learning

Your Profile:

  • motivation and reliability are a prerequisite
  • good knowledge in machine learning and neural networks is an advantage
  • knowledge in python programming

Additional Information:

This thesis project is planned for a duration of 6 months starting immediately. 

Contact:

Franz Pernkopf (pernkopf@tugraz.at)

Wolfgang Roth (roth@tugraz.at)