Articles | Volume 9, issue 2
https://doi.org/10.5194/jsss-9-363-2020
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/jsss-9-363-2020
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Intelligent fault detection of electrical assemblies using hierarchical convolutional networks for supporting automatic optical inspection systems
Alida Ilse Maria Schwebig
CORRESPONDING AUTHOR
Robert Bosch Elektronik GmbH, Salzgitter 38228, Germany
Institut für Produktionsmesstechnik, Technische Universität Braunschweig, Braunschweig 38106, Germany
Rainer Tutsch
Institut für Produktionsmesstechnik, Technische Universität Braunschweig, Braunschweig 38106, Germany
Related authors
Alida Ilse Maria Schwebig and Rainer Tutsch
J. Sens. Sens. Syst., 9, 167–178, https://doi.org/10.5194/jsss-9-167-2020, https://doi.org/10.5194/jsss-9-167-2020, 2020
Short summary
Short summary
This article presents a classification concept based on deep learning as an additional optical test method for real-time visualization and analysis of electrical assemblies in the production environment. For this purpose, a neural convolutional network is used to identify the quality of the solder joint of surface-mounted chip components in the inspection images. The concept can be used to increase the detection performance of the solder joint inspection systems.
Alida Ilse Maria Schwebig and Rainer Tutsch
J. Sens. Sens. Syst., 9, 167–178, https://doi.org/10.5194/jsss-9-167-2020, https://doi.org/10.5194/jsss-9-167-2020, 2020
Short summary
Short summary
This article presents a classification concept based on deep learning as an additional optical test method for real-time visualization and analysis of electrical assemblies in the production environment. For this purpose, a neural convolutional network is used to identify the quality of the solder joint of surface-mounted chip components in the inspection images. The concept can be used to increase the detection performance of the solder joint inspection systems.
Dmytro Sumin and Rainer Tutsch
J. Sens. Sens. Syst., 7, 609–620, https://doi.org/10.5194/jsss-7-609-2018, https://doi.org/10.5194/jsss-7-609-2018, 2018
Short summary
Short summary
This research work suggests a method for thermal effects' compensation during the linear measurements of objects without actual temperature measurements.
Due to certain simplifications, the method is not as accurate as conventional ones (when temperature of the objects and the environment should be measured), but provides much more accuracy than linear measurements without temperature measurements normally do, though maintaining the same pace.
Jan Nitsche, Rolf Kumme, and Rainer Tutsch
J. Sens. Sens. Syst., 7, 577–586, https://doi.org/10.5194/jsss-7-577-2018, https://doi.org/10.5194/jsss-7-577-2018, 2018
Short summary
Short summary
Multi-component sensors for force and moment are commonly used in different areas as robotics, crash tests or material testing. The dynamic behaviour of such sensors may differ significantly from the static behaviour. To analyse the dynamic characteristics, we developed an improved test set-up for periodic excitation. It consists of an electrodynamic shaker, adapting elements and different acceleration references. The first experiments show good results for periodic excitation up to 1000 Hz.
Marc Fischer, Marcus Petz, and Rainer Tutsch
J. Sens. Sens. Syst., 6, 145–153, https://doi.org/10.5194/jsss-6-145-2017, https://doi.org/10.5194/jsss-6-145-2017, 2017
Short summary
Short summary
For optical measurement systems, camera noise is the dominant uncertainty factor when optically cooperative surfaces are measured in a stable environment. In this work it will be shown that the resulting spatial noise can be estimated by means of a noise model. The capability of this approach will be demonstrated for a fringe projection system. This provides a valuable tool for a statistical comparison of different evaluation strategies, which is shown for two different triangulation procedures.
B. Komander, D. Lorenz, M. Fischer, M. Petz, and R. Tutsch
J. Sens. Sens. Syst., 3, 281–290, https://doi.org/10.5194/jsss-3-281-2014, https://doi.org/10.5194/jsss-3-281-2014, 2014
Short summary
Short summary
Deflectometric measurements of specular surfaces when performed with two reference planes lead to both data for surface points and surface normals. The deviations in the surface points are usually several orders of magnitude larger than those for the surface normals. In this paper we propose a method to fuse these data to increase the accuracy of the surface points. The results show that the accuracy can be increased by more than an order of magnitude.
Related subject area
Measurement systems: Sensor signal processing and electronics
Simulation-based investigation of the metrological interface structural resolution capability of X-ray computed tomography scanners
Design of a CMOS memristor emulator-based, self-adaptive spiking analog-to-digital data conversion as the lowest level of a self-x hierarchy
Smart in-cylinder pressure sensor for closed-loop combustion control
Efficient transient testing procedure using a novel experience replay particle swarm optimizer for THD-based robust design and optimization of self-X sensory electronics in industry 4.0
Measurement uncertainty analysis of field-programmable gate-array-based, real-time signal processing for ultrasound flow imaging
Compilation of training datasets for use of convolutional neural networks supporting automatic inspection processes in industry 4.0 based electronic manufacturing
Employing correlation for wireless components and device characterization in reverberation chambers
Impedance model for a high-temperature ceramic humidity sensor
Non-destructive testing of arbitrarily shaped refractive objects with millimetre-wave synthetic aperture radar imaging
Determining the dimension of subsurface defects by active infrared thermography – experimental research
Inverse calculation of strain profiles from ETDR measurements using artificial neural networks
An ultra-low noise capacitance to voltage converter for sensor applications in 0.35 µm CMOS
Development of a non-contact torque transducer based on the laser speckle contrast method
Development of a chopper charge amplifier for measuring the cavity pressure inside injection moulding tools and signal optimisation with a Kalman filter
Optimized mixed-domain signal synthesis for broadband impedance spectroscopy measurements on lithium ion cells for automotive applications
High-accuracy current measurement with low-cost shunts by means of dynamic error correction
A compact readout platform for spectral-optical sensors
High-speed camera-based measurement system for aeroacoustic investigations
Shutter-less calibration of uncooled infrared cameras
A novel approach for detecting HMDSO poisoning of metal oxide gas sensors and improving their stability by temperature cycled operation
Micro- and nanocoordinate measurements of micro-parts with 3-D tunnelling current probing
An electrical characterisation system for the real-time acquisition of multiple independent sensing parameters from organic thin film transistors
Piezoelectric transceiver matching for multiple frequencies
Metal oxide semiconductor gas sensor self-test using Fourier-based impedance spectroscopy
Work area monitoring in dynamic environments using multiple auto-aligning 3-D sensors
Principal component analysis for fast and automated thermographic inspection of internal structures in sandwich parts
Near-field interrogation of SAW resonators on rotating machinery
Magnetic properties of CHAMP and their effects on in-orbit calibration
Matthias Busch and Tino Hausotte
J. Sens. Sens. Syst., 12, 1–8, https://doi.org/10.5194/jsss-12-1-2023, https://doi.org/10.5194/jsss-12-1-2023, 2023
Short summary
Short summary
The paper presents a specimen and evaluation method for interface structural resolution testing of X-ray computed tomography systems based on CT simulations. The geometry is oriented to the norms of image quality investigation. The evaluation is based on profile lines of the cross-sectional images that pass through the boreholes. All statements on resolution refer to the entire measurement chain and, thus, in addition to the reconstruction algorithm also to the surface determination method used.
Hamam Abd and Andreas König
J. Sens. Sens. Syst., 11, 233–262, https://doi.org/10.5194/jsss-11-233-2022, https://doi.org/10.5194/jsss-11-233-2022, 2022
Short summary
Short summary
We pursue a promising novel self-adaptive spiking neural analog-to-digital data conversion (SN-ADC) design that uses spike time to carry information. Thus, SN-ADC can be effectively translated to aggressive new technologies to implement reliable advanced sensory electronic systems. The SN-ADC supports self-x (self-calibration, self-optimization, and self-healing) and machine learning required for the internet of things and Industry 4.0 and is based on a self-adaptive CMOS memristor.
Dennis Vollberg, Peter Gibson, Günter Schultes, Hans-Werner Groh, and Thomas Heinze
J. Sens. Sens. Syst., 11, 1–13, https://doi.org/10.5194/jsss-11-1-2022, https://doi.org/10.5194/jsss-11-1-2022, 2022
Short summary
Short summary
We developed a smart in-cylinder pressure sensor for closed-loop combustion control. The sensor concept is based on a robust and reliable steel membrane equipped with highly strain-sensitive and temperature-stable thin films. The sensor system is complemented by a smart electronics allowing real-time data processing for calculation of different combustion parameters. The data are utilized to control the igniting timing of a spark plug for efficient operation of a combustion engine.
Qummar Zaman, Senan Alraho, and Andreas König
J. Sens. Sens. Syst., 10, 193–206, https://doi.org/10.5194/jsss-10-193-2021, https://doi.org/10.5194/jsss-10-193-2021, 2021
Short summary
Short summary
A novel experience replay particle swarm optimization algorithm is presented and successfully deployed to improve the optimization performance for reconfigurable analog integrated circuits of industry 4.0. An optimization approach is introduced that relied on THD-based indirect measurement method that varies from the traditional calibration approach. Instead, the proposed calibration methodology optimizes all characteristics of the reconfigurable amplifier at once.
Richard Nauber, Lars Büttner, and Jürgen Czarske
J. Sens. Sens. Syst., 9, 227–238, https://doi.org/10.5194/jsss-9-227-2020, https://doi.org/10.5194/jsss-9-227-2020, 2020
Short summary
Short summary
To improve the energy efficiency of industrial processes, such as continuous steel casting, lab-scale experiments with liquid metals are performed. We present an ultrasound Doppler signal process for multiplane, high-frame-rate flow imaging. We achieve real-time operations with an efficient implementation in a field-programmable gate array (FPGA). We analyze the contributions to the uncertainty of the velocity measurement and relate it to its fundamental limit.
Alida Ilse Maria Schwebig and Rainer Tutsch
J. Sens. Sens. Syst., 9, 167–178, https://doi.org/10.5194/jsss-9-167-2020, https://doi.org/10.5194/jsss-9-167-2020, 2020
Short summary
Short summary
This article presents a classification concept based on deep learning as an additional optical test method for real-time visualization and analysis of electrical assemblies in the production environment. For this purpose, a neural convolutional network is used to identify the quality of the solder joint of surface-mounted chip components in the inspection images. The concept can be used to increase the detection performance of the solder joint inspection systems.
Christoph Cammin, Dmytro Krush, Ralf Heynicke, and Gerd Scholl
J. Sens. Sens. Syst., 8, 185–194, https://doi.org/10.5194/jsss-8-185-2019, https://doi.org/10.5194/jsss-8-185-2019, 2019
Short summary
Short summary
Reverberation chambers are well-proven test environments for RF measurements. Typically, mean values are taken from the measurements to characterize the equipment under test. In the novel approach presented in this paper, the correlation of measured sample sequences is utilized to detect deviations, in particular of the radiation characteristics, from reference equipment.
Henrik Lensch, Manuel Bastuck, Tobias Baur, Andreas Schütze, and Tilman Sauerwald
J. Sens. Sens. Syst., 8, 161–169, https://doi.org/10.5194/jsss-8-161-2019, https://doi.org/10.5194/jsss-8-161-2019, 2019
Short summary
Short summary
The measurement of humidity in industrial applications is still an important research issue. Especially under rough operation conditions the current humidity sensor comes to its limitations. To this end, we are developing an integrated sensor system using a metal oxide sensor with impedance spectroscopy as multi-signal generation allowing the discrimination of humidity and reducing gases. The submitted paper focuses on the modeling of the humidity-dependent aspects of impedance.
Ingrid Ullmann, Julian Adametz, Daniel Oppelt, Andreas Benedikter, and Martin Vossiek
J. Sens. Sens. Syst., 7, 309–317, https://doi.org/10.5194/jsss-7-309-2018, https://doi.org/10.5194/jsss-7-309-2018, 2018
Short summary
Short summary
Millimetre-wave radar is an emerging technique for non-destructive testing: it is less costly than X-rays and has no ionizing radiation. Compared to ultrasound it does not require the device under test to be immersed in water. This article presents methods for high-resolution millimetre-wave radar imaging without a priori knowledge of the object's shape. A polymer device with simulated material defects is examined, which could be depicted precisely.
Slawomir Grys
J. Sens. Sens. Syst., 7, 153–160, https://doi.org/10.5194/jsss-7-153-2018, https://doi.org/10.5194/jsss-7-153-2018, 2018
Robin Höhne, Pawel Kostka, and Niels Modler
J. Sens. Sens. Syst., 6, 389–394, https://doi.org/10.5194/jsss-6-389-2017, https://doi.org/10.5194/jsss-6-389-2017, 2017
Short summary
Short summary
This paper focuses on a novel carbon fibre sensor technology that exploits the low-cost and low-energy electrical reflectometry method for a spatially resolved strain measurement. The application of artificial neural networks for mapping the measured electrical signal to the existing strain profile is demonstrated. The potential and current limits are highlighted. The sensor is a promising part for the next generation of light-weight structures with operando health monitoring systems.
Alexander Utz, Christian Walk, Norbert Haas, Tatjana Fedtschenko, Alexander Stanitzki, Mir Mokhtari, Michael Görtz, Michael Kraft, and Rainer Kokozinski
J. Sens. Sens. Syst., 6, 285–301, https://doi.org/10.5194/jsss-6-285-2017, https://doi.org/10.5194/jsss-6-285-2017, 2017
Alberto Garinei and Roberto Marsili
J. Sens. Sens. Syst., 6, 253–258, https://doi.org/10.5194/jsss-6-253-2017, https://doi.org/10.5194/jsss-6-253-2017, 2017
Short summary
Short summary
When a torque measurement is required, torque transducers show many drawbacks during their use: the usual limits are the need for contact and the effects on shaft line parameters. A new approach is proposed in this work: a non-contact torque meter for a machine shaft has been developed, based on a laser speckle contrast method. It carries out torque measurements evaluating the torsional displacement between two distinct sections of the shaft, through the monitoring of their roughness.
Manuel Schneider, Alexander Jahn, Norbert Greifzu, and Norbert Fränzel
J. Sens. Sens. Syst., 6, 199–210, https://doi.org/10.5194/jsss-6-199-2017, https://doi.org/10.5194/jsss-6-199-2017, 2017
Short summary
Short summary
This article describes a chopper amplifier which has been specially developed for piezoelectric pressure sensors. It is shown that the amplifier provides good results for pressure measurement in injection moulds. A special feature of this work is the signal optimisation through the use of a Kalman filter.
Peter Haußmann and Joachim Melbert
J. Sens. Sens. Syst., 6, 65–76, https://doi.org/10.5194/jsss-6-65-2017, https://doi.org/10.5194/jsss-6-65-2017, 2017
Short summary
Short summary
A method to generate broadband waveforms with low peak-to-peak amplitude based on distinct frequency domain profiles is presented, which includes random phase variation and numerical optimization. The peak-to-peak amplitude can be reduced by 71 % compared to zero-phase pulses with equal frequency domain characteristics. Used as excitation signals for impedance spectroscopy, the waveforms yield good results at significantly reduced measurement duration compared to established measurement methods.
Patrick Weßkamp and Joachim Melbert
J. Sens. Sens. Syst., 5, 389–400, https://doi.org/10.5194/jsss-5-389-2016, https://doi.org/10.5194/jsss-5-389-2016, 2016
Short summary
Short summary
Measurement of electrical current is important for many scientific and industrial applications. Often shunt resistors are used. However, thermal effects due to self-heating and ambient temperature variation limit the achievable accuracy.
In this work, a dynamic compensation method is presented which takes static and dynamic temperature drift effects into account. It significantly reduces the remaining measurement errors. The approach can also be used to improve existing measurement systems.
Roland Wuchrer, Sabrina Amrehn, Luhao Liu, Thorsten Wagner, and Thomas Härtling
J. Sens. Sens. Syst., 5, 157–163, https://doi.org/10.5194/jsss-5-157-2016, https://doi.org/10.5194/jsss-5-157-2016, 2016
Short summary
Short summary
The continuous monitoring of industrial and environmental processes is becoming an increasingly important aspect with both economic and societal impact. To realize this task, spectral-optical sensors with their outstanding properties in terms of sensitivity and reliability are a potential solution. However, for exploiting these sensors in a cost- and energy-efficient sensing system, a miniaturized readout unit is needed. Here we present a card-size, inexpensive, and robust readout electronics.
Johannes Gürtler, Daniel Haufe, Anita Schulz, Friedrich Bake, Lars Enghardt, Jürgen Czarske, and Andreas Fischer
J. Sens. Sens. Syst., 5, 125–136, https://doi.org/10.5194/jsss-5-125-2016, https://doi.org/10.5194/jsss-5-125-2016, 2016
Short summary
Short summary
The interaction of sound and flow enables an efficient noise damping. Understanding this aeroacoustic damping phenomenon requires simultaneous measurement of flow and sound fields. Using a high-speed CMOS camera, two-component flow velocity measurements are performed in a three-dimensional region of interest. The sensor system can simultaneously capture sound and turbulent flow velocity oscillations. The presented measurements reveal that the sound energy is transferred into flow energy.
A. Tempelhahn, H. Budzier, V. Krause, and G. Gerlach
J. Sens. Sens. Syst., 5, 9–16, https://doi.org/10.5194/jsss-5-9-2016, https://doi.org/10.5194/jsss-5-9-2016, 2016
Short summary
Short summary
Infrared cameras based on microbolometer focal plane arrays (FPAs) are the most widely used cameras in thermography. For acceptable measurement uncertainty values, the disturbing influences of changing ambient temperature have to be considered. We propose a TEC-less and shutter-less correction approach based on additional temperature measurements inside the IR camera. The effects on the pixel responsivity and offset voltage are considered separately.
M. Schüler, T. Sauerwald, and A. Schütze
J. Sens. Sens. Syst., 4, 305–311, https://doi.org/10.5194/jsss-4-305-2015, https://doi.org/10.5194/jsss-4-305-2015, 2015
Short summary
Short summary
We study the effect of HMDSO vapor on an SnO2-based gas sensor in temperature cycled operation (TCO). The poisoning can be quantified at early stages with a resolution of ±85 ppm*min using TCO. This approach provides a simple method for early detection of HMDSO poisoning.
The stability of gas discrimination by linear discriminant analysis (LDA) can be improved using normalization, which in turn facilitates a more accurate determination of the poisoning state by hierarchical LDA discrimination.
A. Schuler, T. Hausotte, and Z. Sun
J. Sens. Sens. Syst., 4, 199–208, https://doi.org/10.5194/jsss-4-199-2015, https://doi.org/10.5194/jsss-4-199-2015, 2015
Short summary
Short summary
Measurement tasks of modern micro- and nanometrology with decreasing structure sizes and rising aspect ratios require nanometre-resolving 3-D capable sensors. A 3-D probing system based on electrical interaction is presented which is operated on a nanopositioning system NMM-1. New 3-D measurement procedures for the probing system as well as for micrometrology measurement tasks in general are shown.
A. Dragoneas, L. Hague, and M. Grell
J. Sens. Sens. Syst., 4, 169–177, https://doi.org/10.5194/jsss-4-169-2015, https://doi.org/10.5194/jsss-4-169-2015, 2015
A. Ens and L. M. Reindl
J. Sens. Sens. Syst., 4, 9–16, https://doi.org/10.5194/jsss-4-9-2015, https://doi.org/10.5194/jsss-4-9-2015, 2015
M. Schüler, T. Sauerwald, and A. Schütze
J. Sens. Sens. Syst., 3, 213–221, https://doi.org/10.5194/jsss-3-213-2014, https://doi.org/10.5194/jsss-3-213-2014, 2014
Y. Wang, D. Ewert, T. Meisen, D. Schilberg, and S. Jeschke
J. Sens. Sens. Syst., 3, 113–120, https://doi.org/10.5194/jsss-3-113-2014, https://doi.org/10.5194/jsss-3-113-2014, 2014
D. Griefahn, J. Wollnack, and W. Hintze
J. Sens. Sens. Syst., 3, 105–111, https://doi.org/10.5194/jsss-3-105-2014, https://doi.org/10.5194/jsss-3-105-2014, 2014
J.-M. Boccard, P. Katus, R. Renevier, L. M. Reindl, and J.-M. Friedt
J. Sens. Sens. Syst., 2, 147–156, https://doi.org/10.5194/jsss-2-147-2013, https://doi.org/10.5194/jsss-2-147-2013, 2013
H. Lühr, F. Yin, and R. Bock
J. Sens. Sens. Syst., 2, 9–17, https://doi.org/10.5194/jsss-2-9-2013, https://doi.org/10.5194/jsss-2-9-2013, 2013
Cited articles
Berger, M.: Test- und Prüfverfahren in der Elektronikfertigung: Vom
Arbeitsprinzip bis Design-for-Test-Regeln, VDE-Verlag, Berlin, 250 pp., 2012.
Combet, C. and Chang, M.-M.: 01005 Assembly, the AOI route to optimizing
yield, Vi TECHNOLOGY, available at: https://smtnet.com/library/files/upload/01005Assembly.pdf (last access: May 2020), 2009.
Goodfellow, I., Bengio, Y., and Courville, A.: Deep learning, MIT Press,
Cambridge, Massachusetts, London, UK, 1785 pp., 2016.
Hope, T., Resheff, Y. S., and Lieder, I.: Einführung in TensorFlow:
Deep-Learning-Systeme programmieren, trainieren, skalieren und deployen,
Safari Tech Books Online, O'Reilly, Heidelberg, 224 pp., 2018.
Mao, X., Hijazi, S., Casas, R., Kaul, P., Kumar, R., and Rowen, C.:
Hierarchical CNN for traffic sign recognition, in: IEEE Intelligent Vehicles
Symposium (IV), Gothenburg, 130–135, https://doi.org/10.1109/IVS.2016.7535376, 2016.
Müller, A. C. and Guido, S.: Einführung in Machine Learning mit Python: Praxiswissen Data Science, O'Reilly, Heidelberg, 362 pp., 2017.
Niemann, J., Härter, S., Kästle, C., and Franke, J.: Challenges of
the Miniaturization in the Electronics Production on the example of 01005 Components, in: IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile, 113–123, https://doi.org/10.1007/978-3-662-54441-9_12, 2017.
Raschka, S. and Mirjalili, V.: Machine Learning mit Python und Scikit-Learn
und TensorFlow: Das umfassende Praxis-Handbuch für Data Science, Deep
Learning und Predictive Analytics, 2. aktualisierte und erweiterte Auflage,
mitp, Frechen, 577 pp., 2018.
Schwebig, A. I. M. and Tutsch, R.: Compilation of training datasets for use of convolutional neural networks supporting automatic inspection processes in industry 4.0 based electronic manufacturing, J. Sens. Sens. Syst., 9, 167–178, https://doi.org/10.5194/jsss-9-167-2020, 2020.
Silla, C. N. and Freitas, A. A.: A survey of hierarchical classification
across different application domains, Data Min. Knowl. Disc., 22, 31–72,
https://doi.org/10.1007/s10618-010-0175-9, 2011.
Yan, Z., Zhang, H., Piramuthu, R., Jagadeesh, V., DeCoste, D., Wei, D., and
Yu, Y.: HD-CNN: Hierarchical Deep Convolutional Neural Network for Large
Scale Visual Recognition, available at preprint: http://arxiv.org/pdf/1410.0736v4 (last access: May 2020), 2014.
Zheng, Y., Fan, J., Zhang, J., and Gao, X.: Discriminative Fast Hierarchical Learning for Multiclass Image Classification, IEEE T. Neural Netw. Learn. Syst., 31, 2779–2790, https://doi.org/10.1109/TNNLS.2019.2948881, 2020.
Short summary
In order to further increase the performance of neural networks in the field of optical quality assurance of soldered joints, a hierarchical classifier can be used instead of a single network. The global expansion of the classifier enables the inspection task to be distributed over several subnetworks, which results in higher accuracy. Since the individual sub-models only concentrate on the identification of certain characteristics, categorical problems can be solved more effectively.
In order to further increase the performance of neural networks in the field of optical quality...