Development of format applications for information and modeling system environmental monitoring
DOI:
https://doi.org/10.26577/JMMCS-2018-4-556Keywords:
data, NetCDF, database management systems, database, environmental monitoring.Abstract
The need to develop an application format based on modern technologies is determined by the relevance of the optimal functioning of the environmental monitoring system. This system consists of several interacting modules. One of them is the "data assimilation module", which simulates the process using the data assimilation algorithm. For optimal modeling, you need source files in the Network Common Data Form (NetCDF) format, which will be generated from the developed application. The paper describes the characteristics, features, structure of the NetCDF format, presents the existing utilities and software, tools, visualization programs for working with this format. Their purpose, functions are analyzed. The generated NetCDF les are shown using the Scilab math program, which is an alternative to MATLAB. The paper presents a developed database of air pollutants. The structure of the main tables is described: fields, data type, field description, field properties. Database management system (DBMS) MS SQL is used for creating the database. The relational model of created database is presented. The interaction of the main components of the information-modeling system is shown.
References
[1] How to make a website. Unidata. NetCDF. Network Common Data Form NetCdf, Accessed August 20, 2018,
http://website.fishes-aquarium.ru.
[2] Beloushko K.E. Format NetCDF kak standart dlya obmena dannymi v atmosfernyh issledovaniyah [NetCDF format as a standard for data exchange in atmospheric research]. - 2018.- Accessed August 20, 2018, http://www.izmiran.ru/POLAR2012/REPORTS/POLAR_2012_Belouschko2.pdf.
[3] Bartz C., Chasapis K., Kuhn M., Nerge P., Ludwig T. "A best practice analysis of HDF5 and NETCDF-4 using lustre,"Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9137 LNCS (2015): 274-281.
[4] Yu J., Car N.J., Leadbetter A., Simons B.A., Cox, S.J. "Towards linked data conventions for delivery of environmental data using netCDF,"IFIP Advances in Information and Communication Technology, 448 (2015): 102-112.
[5] Car N.J., Ip A., Druken K. "NetCDF-LD SKOS: Demonstrating linked data vocabulary use within NetCDF-compliant files,"IFIP Advances in Information and Communication Technology 507 (2017): 329-337.
[6] Zhuang K., North G.R., Stevens M.J. "A NetCDF version of the two-dimensional energy balance model based on the full multigrid algorithm,"SoftwareX 6 (2017): 198-202.
[7] Visheratin A.A., Rakitin S.V., Ageev D.V., Nasonov D., Boukhanovsky A.V. "Exarch -semantics-based modular distributed data storage,"ACM International Conference Proceeding Series (2017): 22-26.
[8] Scarponi P., Coro G., Pagano P. "A collection of Aquamaps native layers in NetCDF format,"Data in Brief 17 (2018): 292-296.
[9] Hossain M.M, Wu R., Painumkal J.T., Kettouch M., Luca C., Dascalu S.M., Harris F.C. "Web-service framework for environmental models,"Internet Technologies and Applications, ITA 2017 - Proceedings of the 7th International Conference (2017): 104-109.
[10] Rew R., Davis G. "NetCDF: An Interface for Scientific Data Access,"IEEE Computer Graphics and Applications 10(4) (1990): 76-82.
[11] Rew R.K., Hartnett E.J. "Merging NET CDF and HDF5,"Proceedings of the 21st International Conference on Interactive Information Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology (2004): 1-11.
[12] Wingo S.M., Petersen W.A., Gatlin P.N., Pabla C.S., Marks, D.A. "The System for Integrating Multiplatform Data to Build the Atmospheric Column (SIMBA) precipitation observation fusion framework,"Journal of Atmospheric and Oceanic Technology 35 (2018): 1353-1374.
[13] Liu H., Van Oosterom P., Tijssen T., Commandeur T., Wang W. "Managing large multidimensional hydrologic datasets: A case study comparing NetCDF and SciDB,"Journal of Hydroinformatics 20 (2018): 1058-1070.
[14] Mbogo G.-K., Rakitin S.V., Visheratin A. "High-performance meteorological data processing framework for real-time analysis and visualization,"Procedia Computer Science 119 (2017): 334-340.
[15] Husar R.B., Hoijarvi K. "DataFed: Mediated web services for distributed air quality data access and processing,"International Geoscience and Remote Sensing Symposium (IGARSS) (2007): 4016-4019.
[16] Kim J., Kwon Y.C., Kim T.-H. "A Scalable High-Performance I/O System for a Numerical Weather Forecast Model on the Cubed-Sphere Grid,"Asia-Pacific Journal of Atmospheric Sciences 54 (2018): 403-412.
[17] Meng J., Ding G., Liu L., Zhang R. "A comparison and validation of atmosphere CO2concentration OCO-2-based observations and TCCON-based observations,"Communications in Computer and Information Science 645 (2016): 356-363.
[18] Young A.H., Knapp K.R., Inamdar A., Hankins W., Rossow W.B. "The International Satellite Cloud Climatology
Project H-Series climate data record product,"Earth System Science Data 10(1) (2018): 583-593.
[19] Loaiza J.M.C., Giuliani G., Fiameni G. "Big-data in climate change models - A novel approach with hadoop mapreduce,"Proceedings - 2017 International Conference on High Performance Computing and Simulation, HPCS 2017 8035056 (2017): 45-50.
[20] Iona A., Theodorou A., Sofianos S., Troupin C., Troupin C., Beckers J.-M. "Mediterranean Sea climatic indices: Monitoring long-term variability and climate changes,"Earth System Science Data 10(4) (2018): 1829-1842.
[21] Chiriaco M., Dupont J.-C., Bastin S., Badosa J., Lopez J., Haeffelin M., Chepfer H., Guzman R. "ReOBS: A new
approach to synthesize long-term multi-variable dataset and application to the SIRTA supersite,"Earth System Science Data 10(2) (2018): 919-940.
[22] Shutler J.D., Land P.E., Piolle J.-F., Woolf D.K., Goddijn-Murphy L., Paul F., Girard-Ardhuin F., Chapron B., Donlon C.J. "FluxEngine: A flexible processing system for calculating atmosphere-ocean carbon dioxide gas fluxes and climatologies,"Journal of Atmospheric and Oceanic Technology 33(4) (2016): 741-756.
[23] Rodriges Zalipynis R.A. "Array DBMS in environmental science: Satellite sea surface height data in the cloud,"Proceedings of the 2017 IEEE 9th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, IDAACS 2017 2(8095248) (2017): 1062-1065.
[24] ArcGIS for Desktop. Bazovye svedeniya o hranilishche dannyh NetCdf [Basic information about data storage
NetCdf] (2016), Accessed August 21, 2018, http://desktop.arcgis.com/ru/arcmap/10.3/manage-data/netcdf/what-isnetcdf-data.htm.
[25] Rew R., Davis G., Emmerson S., Davies H., Hartne E. The NetCDF Users Guide (2009), Accessed August 21, 2018, https://www.unidata.ucar.edu/software/netcdf//old_docs/docs_4_0_1/netcdf.pdf.
http://website.fishes-aquarium.ru.
[2] Beloushko K.E. Format NetCDF kak standart dlya obmena dannymi v atmosfernyh issledovaniyah [NetCDF format as a standard for data exchange in atmospheric research]. - 2018.- Accessed August 20, 2018, http://www.izmiran.ru/POLAR2012/REPORTS/POLAR_2012_Belouschko2.pdf.
[3] Bartz C., Chasapis K., Kuhn M., Nerge P., Ludwig T. "A best practice analysis of HDF5 and NETCDF-4 using lustre,"Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9137 LNCS (2015): 274-281.
[4] Yu J., Car N.J., Leadbetter A., Simons B.A., Cox, S.J. "Towards linked data conventions for delivery of environmental data using netCDF,"IFIP Advances in Information and Communication Technology, 448 (2015): 102-112.
[5] Car N.J., Ip A., Druken K. "NetCDF-LD SKOS: Demonstrating linked data vocabulary use within NetCDF-compliant files,"IFIP Advances in Information and Communication Technology 507 (2017): 329-337.
[6] Zhuang K., North G.R., Stevens M.J. "A NetCDF version of the two-dimensional energy balance model based on the full multigrid algorithm,"SoftwareX 6 (2017): 198-202.
[7] Visheratin A.A., Rakitin S.V., Ageev D.V., Nasonov D., Boukhanovsky A.V. "Exarch -semantics-based modular distributed data storage,"ACM International Conference Proceeding Series (2017): 22-26.
[8] Scarponi P., Coro G., Pagano P. "A collection of Aquamaps native layers in NetCDF format,"Data in Brief 17 (2018): 292-296.
[9] Hossain M.M, Wu R., Painumkal J.T., Kettouch M., Luca C., Dascalu S.M., Harris F.C. "Web-service framework for environmental models,"Internet Technologies and Applications, ITA 2017 - Proceedings of the 7th International Conference (2017): 104-109.
[10] Rew R., Davis G. "NetCDF: An Interface for Scientific Data Access,"IEEE Computer Graphics and Applications 10(4) (1990): 76-82.
[11] Rew R.K., Hartnett E.J. "Merging NET CDF and HDF5,"Proceedings of the 21st International Conference on Interactive Information Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology (2004): 1-11.
[12] Wingo S.M., Petersen W.A., Gatlin P.N., Pabla C.S., Marks, D.A. "The System for Integrating Multiplatform Data to Build the Atmospheric Column (SIMBA) precipitation observation fusion framework,"Journal of Atmospheric and Oceanic Technology 35 (2018): 1353-1374.
[13] Liu H., Van Oosterom P., Tijssen T., Commandeur T., Wang W. "Managing large multidimensional hydrologic datasets: A case study comparing NetCDF and SciDB,"Journal of Hydroinformatics 20 (2018): 1058-1070.
[14] Mbogo G.-K., Rakitin S.V., Visheratin A. "High-performance meteorological data processing framework for real-time analysis and visualization,"Procedia Computer Science 119 (2017): 334-340.
[15] Husar R.B., Hoijarvi K. "DataFed: Mediated web services for distributed air quality data access and processing,"International Geoscience and Remote Sensing Symposium (IGARSS) (2007): 4016-4019.
[16] Kim J., Kwon Y.C., Kim T.-H. "A Scalable High-Performance I/O System for a Numerical Weather Forecast Model on the Cubed-Sphere Grid,"Asia-Pacific Journal of Atmospheric Sciences 54 (2018): 403-412.
[17] Meng J., Ding G., Liu L., Zhang R. "A comparison and validation of atmosphere CO2concentration OCO-2-based observations and TCCON-based observations,"Communications in Computer and Information Science 645 (2016): 356-363.
[18] Young A.H., Knapp K.R., Inamdar A., Hankins W., Rossow W.B. "The International Satellite Cloud Climatology
Project H-Series climate data record product,"Earth System Science Data 10(1) (2018): 583-593.
[19] Loaiza J.M.C., Giuliani G., Fiameni G. "Big-data in climate change models - A novel approach with hadoop mapreduce,"Proceedings - 2017 International Conference on High Performance Computing and Simulation, HPCS 2017 8035056 (2017): 45-50.
[20] Iona A., Theodorou A., Sofianos S., Troupin C., Troupin C., Beckers J.-M. "Mediterranean Sea climatic indices: Monitoring long-term variability and climate changes,"Earth System Science Data 10(4) (2018): 1829-1842.
[21] Chiriaco M., Dupont J.-C., Bastin S., Badosa J., Lopez J., Haeffelin M., Chepfer H., Guzman R. "ReOBS: A new
approach to synthesize long-term multi-variable dataset and application to the SIRTA supersite,"Earth System Science Data 10(2) (2018): 919-940.
[22] Shutler J.D., Land P.E., Piolle J.-F., Woolf D.K., Goddijn-Murphy L., Paul F., Girard-Ardhuin F., Chapron B., Donlon C.J. "FluxEngine: A flexible processing system for calculating atmosphere-ocean carbon dioxide gas fluxes and climatologies,"Journal of Atmospheric and Oceanic Technology 33(4) (2016): 741-756.
[23] Rodriges Zalipynis R.A. "Array DBMS in environmental science: Satellite sea surface height data in the cloud,"Proceedings of the 2017 IEEE 9th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, IDAACS 2017 2(8095248) (2017): 1062-1065.
[24] ArcGIS for Desktop. Bazovye svedeniya o hranilishche dannyh NetCdf [Basic information about data storage
NetCdf] (2016), Accessed August 21, 2018, http://desktop.arcgis.com/ru/arcmap/10.3/manage-data/netcdf/what-isnetcdf-data.htm.
[25] Rew R., Davis G., Emmerson S., Davies H., Hartne E. The NetCDF Users Guide (2009), Accessed August 21, 2018, https://www.unidata.ucar.edu/software/netcdf//old_docs/docs_4_0_1/netcdf.pdf.
Downloads
How to Cite
Iskakov, K. T., Khassenova Z.T., Z. T., Mynbayeva, B. N., Oralbekova, Z. O., & Mukhtarova, S. O. (2019). Development of format applications for information and modeling system environmental monitoring. Journal of Mathematics, Mechanics and Computer Science, 100(4), 52–62. https://doi.org/10.26577/JMMCS-2018-4-556
Issue
Section
Computer Science
