Authors:
László Vidács MTA-SZTE Mesterséges Intelligencia Kutatócsoport [University of Szeged and Hungarian Academy of Sciences, MTA-SZTE Research Group on Artificial Intelligence] Szeged Hungary

Search for other papers by László Vidács in
Current site
Google Scholar
PubMed
Close
,
Márk Jelasity Szegedi Tudományegyetem Számítógépes Algoritmusok és Mesterséges Intelligencia Tanszék [University of Szeged, Department of Algorithms and AI] Szeged Hungary

Search for other papers by Márk Jelasity in
Current site
Google Scholar
PubMed
Close
,
László Tóth Szegedi Tudományegyetem Szoftverfejlesztési Tanszék [University of Szeged, Department of Software Engineering] Szeged Hungary

Search for other papers by László Tóth in
Current site
Google Scholar
PubMed
Close
,
Péter Hegedűs MTA-SZTE Mesterséges Intelligencia Kutatócsoport [University of Szeged and Hungarian Academy of Sciences, MTA-SZTE Research Group on Artificial Intelligence] Szeged Hungary

Search for other papers by Péter Hegedűs in
Current site
Google Scholar
PubMed
Close
, and
Rudolf Ferenc Szegedi Tudományegyetem Szoftverfejlesztési Tanszék [University of Szeged, Department of Software Engineering] Szeged Hungary

Search for other papers by Rudolf Ferenc in
Current site
Google Scholar
PubMed
Close
Open access

Összefoglalás. A mély mesterséges neuronhálók elterjedése az ipari alkalmazásokban évekkel azok megbízhatóságával, értelmezhetőségével, és biztonságával kapcsolatos szakterületek fejlődését megelőzően történt. Az egyik, gyakorlatban is jelentős területen, a képfelismerésben például a megvalósult megoldások szinte már emberi teljesítményre képesek, de ezzel együtt célzott zajjal ezek a rendszerek félrevezethetők, megzavarhatók. Jelen kéziratban ismertetünk néhány tipikus biztonsági problémát, valamint rámutatunk arra, hogy a hagyományos szoftverfejlesztés területén alkalmazott minőségbiztosítási módszerekkel rokon megoldásokra szükség van az MI-re épülő rendszerek fejlesztésében, akár a mesterséges neuronhálók biztonságát, akár az MI rendszerek hagyományos komponenseinek fejlesztését tartjuk szem előtt.

Summary. Research on the trustworthiness, interpretability and security of deep neural networks lags behind the widespread application of the technology in industrial applications. For example, in image recognition, modern solutions are capable of nearly human performance. However, with targeted adversarial noise, these systems can be arbitrarily manipulated. Here, we discuss some of the security problems and point out that quality assurance methods used in traditional software development should also be adapted when developing AI-based systems, whether in the security of artificial neural networks or traditional components of AI systems. One of the main concerns about neural networks today that – to the best of our knowledge – affects all deep neural networks is the existence of adversarial examples. These examples are relatively easy to find and according to a recent experiment, a well-chosen input can attack more networks at the same time. In this paper we also present a wider perspective of security of neural architectures borrowed from the traditional software engineering discipline. While in traditional development several methods are widely applied for software testing and fault localization, there is a lack of similar well-established methods in the neural network context. In case of deep neural networks, systematic testing tools and methods are in the early stage, and a methodology to test and verify the proper behavior of the neural networks is highly desirable. Robustness testing of machine learning algorithms is a further issue. This requires the generation of large random input data using fuzz testing methods. The adaptation of automatic fault localization techniques has already started by defining notions like code coverage to neural networks. Lastly, we argue that the effective development of high quality AI-based systems need well suited frameworks that can facilitate the daily work of scientists and software developers – like the Deep-Water framework, presented in the closing part of the paper.

  • Beszédes Á. , Horváth F. , Di Penta M. & Gyimóthy T. , 'Leveraging Contextual Information from Function Call Chains to Improve Fault Localization ' 27th IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER 2020) , , London, Ontario, Canada (accepted for presentation) .

    • Search Google Scholar
    • Export Citation
  • Chen Tsong Yueh , Kuo Fei-Ching , Liu Huai , Poon Pak-Lok , Towey Dave , Tse T. H. & Zhou Zhi Quan , 'Metamorphic Testing ' (2018 ) 51 (1 ) ACM Computing Surveys , 04 .

  • https://www.comet.ml/ Comet website Comet

  • Csuvik Viktor , Horvath Deniel , Horvath Ferenc & Vidacs Laszlo , 'Utilizing Source Code Embeddings to Identify Correct Patches ' 2020 IEEE 2nd International Workshop on Intelligent Bug Fixing (IBF) , , .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Databricks Inc. Mlflow, an open source platform for the machine learning lifecycle 2020 https://mlflow.org/

  • Eniser H.F. , Gerasimou S. & Sen A. R. Hähnle & W. van der Aalst (eds)DeepFault: Fault Localization for Deep Neural Networks Fundamental Approaches to Software Engineering. FASE 2019. Lecture Notes in Computer Science Springer, Cham. 11424 2019

    • Crossref
    • Search Google Scholar
    • Export Citation
  • European Commission (eds) On Artificial Intelligence – A European approach to excellence and trust. White paper COM(2020) 65, Brussels 2020.02.19. https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf 2020.03.04. 2020

    • Search Google Scholar
    • Export Citation
  • Fischetti Matteo & Jo Jason , 'Deep neural networks and mixed integer linear optimization ' (2018 ) 23 (3 ) Constraints , 04 .

  • Forti M. , Nistri P. & Papini D. , 'Global Exponential Stability and Global Convergence in Finite Time of Delayed Neural Networks With Infinite Gain ' (2005 ) 16 (6 ) IEEE Transactions on Neural Networks , 11 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gandy A. & Scott J. , 'Unit Testing for MCMC and other Monte Carlo Methods ' (2020 ) arXiv: Methodology .

  • Greff Klaus , Klein Aaron , Chovanec Martin , Hutter Frank & Schmidhuber Jürgen , 'The Sacred Infrastructure for Computational Research ' (2018 ) figshare .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gleave A. , Dennis M. , Wild C. , Kant N. , Levine S. & Russell S. , 'Adversarial Policies: Attacking Deep Reinforcement Learning ' International Conference on Learning Representations , , .

    • Search Google Scholar
    • Export Citation
  • Hamm Lonnie , Brorsen B. Wade & Hagan Martin T. , 'Comparison of Stochastic Global Optimization Methods to Estimate Neural Network Weights ' (2007 ) 26 (3 ) Neural Processing Letters , 09 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harrold M. J. & Stasko J. , 'Visualization of test information to assist fault localization ' 24th International Conference on Software Engineering, ACM , , .

    • Search Google Scholar
    • Export Citation
  • Hodovan Renata & Kiss Akos , 'Fuzzinator: An Open-Source Modular Random Testing Framework ' 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST) , , .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hodován R. , Vince D. & Kiss Á. , 'Fuzzing JavaScript Environment APIs with Interdependent Function Calls ' Integrated Formal Methods – 15th International Conference, IFM 2019 , , Bergen, Norway .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Islam J. , Pan R. , Nguyen G. & Rajan H. , 'Repairing Deep Neural Networks: Fix Patterns and Challenges ' ICSE’20: The 42nd International Conference on Software Engineering , , .

    • Search Google Scholar
    • Export Citation
  • Megyeri I. , Hegedűs I. & Jelasity M. , 'Adversarial Robustness of Model Sets ' Proceedings of the International Joint Conference on Neural Networks (IJCNN) , , .

    • Search Google Scholar
    • Export Citation
  • Azure Machine Learning Studio Microsoft 2018 https://studio.azureml.net/

  • Ehmer Mohd & Khan Farmeena , 'A Comparative Study of White Box, Black Box and Grey Box Testing Techniques ' (2012 ) 3 (6 ) International Journal of Advanced Computer Science and Applications .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Polyaxon an open source platform for reproducible machine learning at scale POLYAXON 2019 https://polyaxon.com/

  • Rauber Paulo E. , Fadel Samuel G. , Falcao Alexandre X. & Telea Alexandru C. , 'Visualizing the Hidden Activity of Artificial Neural Networks ' (2017 ) 23 (1 ) IEEE Transactions on Visualization and Computer Graphics , 01 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ferenc Rudolf , Viszkok Tamás , Aladics Tamás , Jász Judit & Hegedűs Péter , 'Deep-water framework: The Swiss army knife of humans working with machine learning models ' (2020 ) 12 SoftwareX , 07 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • STUDIO.ML COMMUNITY Studio.ml STUDIO.ML COMMUNITY 2017 https://studioml.readthedocs.io/en/latest/

  • Szegedy C. , Zaremba W. , Sutskever I. , Bruna J. , D. Erhan D. , Goodfellow I. J. & Fergus R. , 'Intriguing properties of neural networks ' 2nd International Conference on Learning Representations (ICLR) , , .

    • Search Google Scholar
    • Export Citation
  • Tarlow D. , Moitra S. , Rice A. , Chen Z. , Manzagol P-A. , Sutton C. & Aftandilian E. , 'Learning to Fix Build Errors with Graph2Diff Neural Networks ' 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops (ICSEW) , , Seoul, Republic of Korea .

    • Search Google Scholar
    • Export Citation
  • University of Szeged Department of Software Engineering Deep Water Framework University of Szeged Department Of Software Engineering 2019 https://github.com/sed-inf-u-szeged/DeepWaterFramework

    • Search Google Scholar
    • Export Citation
  • Zhu Hong , Liu Dongmei , Bayley Ian , Harrison Rachel & Cuzzolin Fabio , 'Datamorphic Testing: A Method for Testing Intelligent Applications ' 2019 IEEE International Conference On Artificial Intelligence Testing (AITest) , , .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie Xiaofei , See Simon , Ma Lei , Juefei-Xu Felix , Xue Minhui , Chen Hongxu , Liu Yang , Zhao Jianjun , Li Bo & Yin Jianxiong , 'DeepHunter: a coverage-guided fuzz testing framework for deep neural networks ' the 28th ACM SIGSOFT International Symposium , , .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xu Han , Ma Yao , Liu Hao-Chen , Deb Debayan , Liu Hui , Tang Ji-Liang & Jain Anil K. , 'Adversarial Attacks and Defenses in Images, Graphs and Text: A Review ' (2020 ) 17 (2 ) International Journal of Automation and Computing , 03 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun Youcheng , Huang Xiaowei , Kroening Daniel , Sharp James , Hill Matthew & Ashmore Rob , 'Structural Test Coverage Criteria for Deep Neural Networks ' (2019 ) 18 (5s ) ACM Transactions on Embedded Computing Systems , 10 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collapse
  • Expand

Editor-in-Chief:

Founding Editor-in-Chief:

  • Tamás NÉMETH

Managing Editor:

  • István SABJANICS (Ministry of Interior, Budapest, Hungary)

Editorial Board:

  • Attila ASZÓDI (Budapest University of Technology and Economics)
  • Zoltán BIRKNER (University of Pannonia)
  • Valéria CSÉPE (Research Centre for Natural Sciences, Brain Imaging Centre)
  • Gergely DELI (University of Public Service)
  • Tamás DEZSŐ (Migration Research Institute)
  • Imre DOBÁK (University of Public Service)
  • Marcell Gyula GÁSPÁR (University of Miskolc)
  • József HALLER (University of Public Service)
  • Charaf HASSAN (Budapest University of Technology and Economics)
  • Zoltán GYŐRI (Hungaricum Committee)
  • János JÓZSA (Budapest University of Technology and Economics)
  • András KOLTAY (National Media and Infocommunications Authority)
  • Gábor KOVÁCS (University of Public Service)
  • Levente KOVÁCS buda University)
  • Melinda KOVÁCS (Hungarian University of Agriculture and Life Sciences (MATE))
  • Miklós MARÓTH (Avicenna Institue of Middle Eastern Studies )
  • Judit MÓGOR (Ministry of Interior National Directorate General for Disaster Management)
  • József PALLO (University of Public Service)
  • István SABJANICS (Ministry of Interior)
  • Péter SZABÓ (Hungarian University of Agriculture and Life Sciences (MATE))
  • Miklós SZÓCSKA (Semmelweis University)

Ministry of Interior
Science Strategy and Coordination Department
Address: H-2090 Remeteszőlős, Nagykovácsi út 3.
Phone: (+36 26) 795 906
E-mail: scietsec@bm.gov.hu

DOAJ

2023  
CrossRef Documents 32
CrossRef Cites 15
Days from submission to acceptance 59
Days from acceptance to publication 104
Acceptance Rate 81%

2022  
CrossRef Documents 38
CrossRef Cites 10
Days from submission to acceptance 54
Days from acceptance to publication 78
Acceptance Rate 84%

2021  
CrossRef Documents 46
CrossRef Cites 0
Days from submission to acceptance 33
Days from acceptance to publication 85
Acceptance Rate 93%

2020  
CrossRef Documents 13
CrossRef Cites 0
Days from submission to acceptance 30
Days from acceptance to publication 62
Acceptance Rate 93%

Publication Model Gold Open Access
Submission Fee none
Article Processing Charge none

Scientia et Securitas
Language Hungarian
English
Size A4
Year of
Foundation
2020
Volumes
per Year
1
Issues
per Year
4
Founder Academic Council of Home Affairs and
Association of Hungarian PhD and DLA Candidates
Founder's
Address
H-2090 Remeteszőlős, Hungary, Nagykovácsi út 3.
H-1055 Budapest, Hungary Falk Miksa utca 1.
Publisher Akadémiai Kiadó
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
Applied
Licenses
CC-BY 4.0
CC-BY-NC 4.0
ISSN ISSN 2732-2688 (online), 3057-9759 (print)
   

Monthly Content Usage

Abstract Views Full Text Views PDF Downloads
Aug 2024 0 29 7
Sep 2024 0 94 9
Oct 2024 0 257 12
Nov 2024 0 121 11
Dec 2024 0 49 4
Jan 2025 0 65 7
Feb 2025 0 19 0