To the top

Page Manager: Webmaster
Last update: 9/11/2012 3:13 PM

Tell a friend about this page
Print version

Towards Structured Evalua… - University of Gothenburg, Sweden Till startsida
Sitemap
To content Read more about how we use cookies on gu.se

Contact form








 


Note! If you want an answer on a question you must specify your email address




Towards Structured Evaluation of Deep Neural Network Supervisors

Book
Authors J. Henriksson
Christian Berger
M. Borg
L. Tornberg
C. Englund
S. R. Sathyamoorthy
S. Ursing
Ieee, Ieee,
ISBN 978-1-7281-0492-8
Publication year 2019
Published at Department of Computer Science and Engineering (GU)
Language en
Links dx.doi.org/10.1109/AITest.2019.00-1...
Keywords deep neural networks, robustness, out-of-distribution, supervisor, automotive perception, OCEEDINGS7th Industrial Conference on Data Mining, JUL 14-18, 2007, Leipzig, GERMANY, V4597, OCESSING (WCSP)7th International Conference on Wireless Communications and Signal Processing
Subject categories Software Engineering

Abstract

Deep Neural Networks (DNN) have improved the quality of several non-safety related products in the past years. However, before DNNs should be deployed to safety-critical applications, their robustness needs to be systematically analyzed. A common challenge for DNNs occurs when input is dissimilar to the training set, which might lead to high confidence predictions despite proper knowledge of the input. Several previous studies have proposed to complement DNNs with a supervisor that detects when inputs are outside the scope of the network. Most of these supervisors, however, are developed and tested for a selected scenario using a specific performance metric. In this work, we emphasize the need to assess and compare the performance of supervisors in a structured way. We present a framework constituted by four datasets organized in six test cases combined with seven evaluation metrics. The test cases provide varying complexity and include data from publicly available sources as well as a novel dataset consisting of images from simulated driving scenarios. The latter we plan to make publicly available. Our framework can be used to support DNN supervisor evaluation, which in turn could be used to motive development, validation, and deployment of DNNs in safety-critical applications.

Page Manager: Webmaster|Last update: 9/11/2012
Share:

The University of Gothenburg uses cookies to provide you with the best possible user experience. By continuing on this website, you approve of our use of cookies.  What are cookies?