|  Introduction to Big Data Team of Brilliance Technology

As a team engaged in big data technology and its application research, Brilliance Science and Technology Innovation Center has nearly 100 big data products, R & D and implementation teams, and has mature implementation cases and big data products in many fields. Brilliance Big Data Solution fully integrates and utilizes the current smart city, snow bright project, safe city, smart community, smart park, smart city management, smart environmental protection, smart justice, smart transportation and other perception resources of the Internet of Things, based on the e-government extranet, Internet, mobile Internet, industry private network, video private network, etc. Based on the three-dimensional visualization of space-time geographic information and digital twins, it serves the technical architecture of big data applications in various industries. It collects multi-dimensional static and dynamic data such as face recognition, license plate recognition, intelligent access control, traffic entrance, space-time trajectory, mobile MAC, vehicle RFID, IOT identification, personnel and physical organization, business systems of commissions and bureaus, economic industry, social and livelihood, ecological and environmental protection, and sharing and exchange of commissions and bureaus. Realize data collection, data fusion, data governance, data standardization, data modeling, intelligent algorithm optimization, data mining analysis and data visualization. Provide big data analysis services for comprehensive social governance, economic industry, macroeconomic development, industrial integration, resource consumption, GDP output, KPI life index, government services, poverty alleviation big data, ecological environmental protection, people's livelihood education and other industries.

| Solution Benefits

Brilliance Big Data Service not only provides technical development services based on big data, but also provides a complete set of solutions and technical consulting services, covering all aspects from infrastructure layer to appication layer construction.

1. Support big data collection, governance and data standardization based on the mainstream big data processing framework;

2. Support data mining and data visualization applications based on knowledge mapping and intelligent algorithms;

3. Construct the whole process of data acquisition and fusion, cleaning modeling, mining analysis and data application value of data factory.

In the application layer, Brilliance provides big data solutions for various industries; in the data middle platform layer, it provides process-driven business middle platform and data middle platform dual-platform management to achieve customer-centric omni-channel business data integration; in the data governance layer, it includes all technical frameworks and technical development tool chains related to the underlying technology of big data governance, and supports multi-source heterogeneous data sources; In the platform layer, we provide PaaS services based on the enterprise cloud platform, which can be deployed on any cloud platform.

| Big Data Product Capabilities

●  Data governance process

●  A collection of data

Connect with government departments, enterprises, Internet and other units, support multi-source heterogeneous data, and use different technical means such as Brilliance Exchange Platform, open source and domestic ETL tools, message queues, and specialized interface services and software according to different data sources.

●  Data cleaning and translation

Because of the large amount of data, most of the data quality control is completed in the data preprocessing link, that is, in the ETL conversion, the standard (Ministry standard, national standard, Bureau standard) and code conversion are carried out, the data quality log (empty, illegal field, code correspondence, etc.) Is recorded, the data quality monitoring adopts ELK architecture, and there are special clusters for data quality monitoring. Through KABANA, data preprocessing and data quality logs can be monitored and counted in real time.

The data processing process mainly includes the elimination of ambiguity, the elimination of data duplication, the completion of data, the correction of errors, the unification of code tables, the unification of data formats and so on. After this series of processing, the data has reached the unified standard and format, and there is no ambiguity to prepare for the subsequent data application.

●  Data consanguinity management

According to the source of data, application services and related resources, it can help users quickly grasp where each type of data comes from, where it goes, who it is used for, as well as the relationship between data and other resources, the periodic change of data resources, and the access of resources, so as to realize the visual display of the operation of full data assets.

●  Data governance results and storage

After data standardization governance is completed, the data is stored in the data warehouse.Data governance results are used to support real-time computing/offline computing/knowledge mapping analysis.

Real-time and offline: HBASE and HIVE under the HADOOP system are used, and a separate full-text retrieval cluster is used for full-category and full-volume data indexing.

Knowledge mapping: domestic knowledge mapping database is adopted, with entity quantity (5 billion +) and relational data (50 billion +).

● Data visualization service system

Visual service system realizes the visual development and release of data services. Reduce the difficulty of development and provide an interactive interface between business personnel and data. The main functions include visual development, visual analysis and visual modeling.

●  Offline and real-time computing architecture

Real-time statistical calculation, situation presentation, report, etc. are based on MPP database, and large tables of 10 billion levels can reach sub-second level grouping statistical calculation (including time dimension and any other cross dimension). Tens of billions to tens of millions of multi-table association statistics can also reach the second level (need to do a lot of optimization work).

●  Upper layer application

The big data platform supports more than 50 global applications, covering retrieval, comparison and collision, statistical analysis, data mining, data download, etc., with an average daily access of more than 100 million service interfaces.

●Situation presentation and statistical report

Situation display: mainly through the customized command screen, the chart control is based on open source (ECHARTS).

Statistical report: an autonomous platform for visual customization.

● Visualization analysis of knowledge map

Combined with the graphical analysis software of Brilliance's big data knowledge map, it is widely used in the search of relatives, the search of communication relations, the presentation of hidden relations, the excavation of gangs, and the fight against crime and other financial scenarios.

●  Big data autonomous modeling

Through our company's "Modeling Workshop" software, we provide big data independent modeling capabilities for platform users. The data source is used by dragging and dropping, and the data processing logic and calculation logic are combined by workload, providing common algorithms such as data processing, set operation, regression, clustering, and classification.

| Big Data Middleware Advantage

Brilliance big data middle platform solution provides dual middle platform management of process-driven business middle platform and data middle platform, realizes customer-centered omni-channel business data integration, thereby improving operational efficiency and inventory turnover rate, and promotes rapid business growth. Through the integration of upstream and downstream resources, the overall optimization and reconstruction of the industrial chain will accelerate the digital transformation and upgrading.

As a big data service platform for enterprise applications, it has the following characteristics:

(1) Support multi-source heterogeneous data access, applicable to data access in various scenarios;

(2) Unified data management platform, integrated batch processing and real-time processing engine, to achieve data cleaning, standardization, integration, etc.;

(3) Provide various intelligent retrieval functions, including knowledge retrieval and custom multi-condition retrieval;

(4) Flexible custom knowledge modeling system, which can customize the indication model to meet various application scenarios;

(5) Provide a multi-data analysis tool to perform multi-angle space-time collision analysis on business data;

(6) Integrate multi-service data to form a unified data and service management platform;

(7) Calculation and storage of massive data;

Integrate the popular big data processing framework in the industry to achieve massive data computing and storage.

(8) Support the enterprise cloud platform based on Brilliance, making operation and maintenance easier;

Support the system built on the basis of Brilliance Enterprise Cloud Platform to realize easy operation and maintenance and easy system expansion.

(9) The technology is completely autonomous and controllable;

The technologies and components used are either self-developed by Brilliance (Brilliance has complete intellectual property rights) or third-party open source frameworks, and the overall solution technology is autonomous and controllable.

(10) Business scenarios can be customized;

Support for any business scenario can be provided according to business requirements.

(11) Compatible with various private clouds and public clouds.

The platform does not depend on a specific cloud platform resource, and can use a variety of resources such as physical machines, VMs, and cloud platforms as the infrastructure layer.