Identification of encrypted and malicious network traffic based on one-dimensional convolutional neural network

Abstract

The rapid advancement of the Internet has brought a exponential growth in network traffic. At present, devices deployed at edge nodes process huge amount of data, extract key features of network traffic and then forward them to the cloud server/data center. However, since the efficiency of mobile terminal devices in identifying and classifying encrypted and malicious traffic lags behind, how to identify network traffic more efficiently and accurately remains a challenging problem. We design a convolutional neural network model: One-dimensional convolutional neural network with hexadecimal data (HexCNN-1D) that combines normalized processing and attention mechanisms. By adding the attention mechanism modules Global Attention Block (GAB) and Category Attention Block (CAB), network traffic is classified and identified. By extracting effective load information from hexadecimal network traffic, our model can identify most categories of network traffic including encrypted and malicious traffic data. The experimental results show that the average accuracy is 98.8%. Our model can greatly improve the accuracy of network traffic data recognition.

Introduction

Recent years have witnessed the development of Cloud Computing, the Internet of things (IoTs) and even the conceptual Internet of Everything (IoE) [1, 2], and intelligent application terminals in modern world are also developing in coherence with such trend. How to conduct real-time data analysis with limited networking and computing resources, and how to conduct network traffic analysis and classification via edge\cloud computing devices, which poses new challenges to network traffic monitoring and related issues. Firstly, cybersecurity must possess the capability to identify and block intrusive traffic data [3, 4]. Network security analysis of network traffic mainly involves identification of malicious network traffic to prevent malicious network attacks resulting in significant economic losses [5, 6]. Secondly, identification and classification of network traffic with higher accuracy will improve network Quality of Service (QoS) and enable efficient traffic monitoring followed with effective resource allocation. Thirdly, network traffic identification can also be applied to Industrial Internet, Internet of Things (IoT), and cloud/edge network systems as well [7]. B. He et al. [8] proposed an intelligent VNFs configuration framework to solve the problem of network resource scheduling in CoT. Accurate traffic identification has been recognized as a crucial technology for improving Quality of Service (QoS) of the network [9, 10]. To solve these problems mentioned above, many approached and works focusing on network traffic identification have been proposed. Among these proposed approaches, those based on machine learning have the most promising prospects in general [11]. With the increasing number of terminal devices in the Internet of Things, the need for more efficient use of shared computing and communication resources in an end-to-end edge-cloud environment becomes more urgent [12].

According to the recently published works, the methods for identifying network traffic mostly focused on the training of machine learning models, such as convolutional neural networks, recurrent neural networks, decision trees, etc. Although machine learning models can greatly accelerate the extraction of network traffic characteristics [13], most existing solutions do not take into account the processing of the network traffic data format itself.

Our contributions are as follows:

• 1. In the model data pre-processing process, we introduced the normalization module to resolve the problem of insufficient and unbalanced data distribution caused by the small difference of network traffic categories.

• 2. In the model design process, we introduced the adjusted Global Attention Block (GAB) and Category Attention Block (CAB) to deal with more detailed data information of encrypted traffic and malicious traffic category.

• 3. We designed four different network experimental environments to identify conventional traffic, encrypted traffic, malicious traffic, mixed traffic, etc., so that it can detect and classify different network traffic categories more efficiently. The classification results are compared with other more advanced methods. The results show that our model can identify and classify network traffic data categories with higher precision.

The rest of this paper is structured as follows: The second section discusses the related work. The third section introduces the data preprocessing. The fourth and fifth sections respectively introduce the convolutional neural network model [14] HexCNN-1D and the batch normalization and attention mechanism module we added after adjustment, respectively. The sixth section presents the experimental data and index setting. The seventh section analyzes the experimental results. Section eighth discusses our future work and improvements.

Related work

There are four main network traffic classification methods [15,16,17]: methods based on port identification [18], methods based on deep packet detection [19], methods based on statistical processing [20], and methods based on user behavior [21]. Most network protocol ports are based on security policies; thus, the identification accuracy of such port identification-based methods is relatively low, and the deep packet detection-based method cannot process the current encrypted network traffic data. With the ubiquitous usage of machine learning [22,23,24], researchers are investigating approaches based on statistical processing and behavioral norms.

Traditional network traffic classification methods include clustering, support vector machines, C4.5 Decision Tree (C4.5), and etc. Most of these traditional methods have low accuracy and low classification efficiency. Anshu Priya et al. [25] proposed the use of K-Means clustering algorithm to analyze real-time network data traffic situations in universities. However, the clustering algorithm has poor classification efficiency for data categories with high similarity. Wang et al. [26] used C4.5 to classify P2P traffic, which is used to describe the behaviour characteristics of applications. The disadvantages of the C4.5 algorithm is that the training time is long and it is only suitable for processing small data sets. Coull et al. [27] proposed to classify p2p traffic by analyzing packet features and proposed traffic analysis of encrypted messaging services: Apple iMessage and other message classification. Mauro et al. [28] proposed to uncover encrypted WebRTC traffic by machine learning tools, using the random forest approach. Traditional feature-based statistical classifiers are becoming less suitable for today's massive data processing.

Deep learning has been gradually hybrided with more research fields to generate more efficient and appliable network models thanks to its powerful function extraction capability and efficient model parameter calculation. Segun I. Popoola et al. [4] proposed a deep neural network to classify network traffic in the scenario of Internet of Things, aiming at Zero-Day Botnet Attack Detection. However, the time cost of training model is large, so it cannot be applied to large-scale data. Shi Dong et al. [29] proposed an optimization method for abnormal network traffic detection based on a semi-supervised double-depth Q-network (SSDDQN). Based on the above, Shi Dong [30] proposed an improved support vector machine (SVM) algorithm, the cost-sensitive support vector machine (CMSVM), to solve the imbalance problem in network traffic identification. Wang et al. [31, 32] for feature extraction from raw traffic data after preprocessing in two dimensions of CNN-1D and CNN-2D. The authors demonstrated the superiority of these two methods by observing and elaborating the accuracy scores achieved in the experimental evaluation metrics, etc. Lotfollahi et al. [33] proposed Deep Packet: a new method for cryptographic traffic classification using deep learning. However, the shortcoming of deep packet detection technology is obvious, it is vulnerable against the same kind of network attacks, and the deployment of deep packet detection is difficult, lest additional burden on the processor. Zou et al. [34] proposed a method for cryptographic traffic classification method based on convolutional Long Short-Term Memory (LSTM) neural networks. However, after a long period of training and increasing of the number of layers, the problem of gradient explosion is easily encountered. Bu et al. [35] proposed a deep parallel network (NIN) neural network model. Since its introduction, Deep learning has played an increasingly important role in machine learning. Convolutional neural networks (CNN), recurrent neural networks (RNN), and long and short-term memory network (LSTM) models gained their recognition for their excellent performance in the field of computer vision.

There are many common traffic classification methods, each with its own advantages and disadvantages. For example, port number-based classification is the easiest to implement, but has low identification accuracy and limited applicability. The classification method based on deep packets has a high accuracy but cannot detect encryption services. Therefore, future research will focus on network traffic classification and identification using machine learning methods. As a part of machine learning, researchers are trying to apply deep learning to the field of network traffic recognition technology. In this paper, a lightweight neural network model is proposed to identify classified network traffic data types.

Network traffic data pre-processing

Hexadecimal data of network traffic conversion

The ISCX-VPN-NonVPN-2016 and USTC-TFC2016 datasets are used in this paper. As shown in Table 1, we selected the following nine data streams by category in the ISCX-VPN-NonVPN-2016 dataset: AIM, Facebook, Email, Netflix, Hangouts, YouTube, Skype, Vimeo, and Spotify, and packets corresponding to the nine data steams encapsulated by the VPN.

Table 1 The data used In ISCX-VPN-NonVPN-2016

As shown in Table 2, we selected the 7 + 3 category in the USTC-TFC2016 dataset. Among them, there are seven different types of regular network traffic: BitTorrent, Facetime, Gmail, MySQL, Skype, Weibo, and World of Warcraft, and three different types of malicious network traffic: Zeus, Virut, and Nsis-ay. Above table shows the selected network traffic data types along with volume statistics.

Table 2 The data used In USTC-TFC2016

We find that the effective content output in hexadecimal form in each PACP packet in the two datasets has obvious characteristic features, and most of the effective bytes in the packet are between [50, 1480] bytes.

Therefore, for the data flows captured in the dataset described above, we store approximately 5000 pieces of data in hexadecimal format for each type of data flow. Each data flow collects 1480 bytes of packet load through the preprocessing model. If the payload length is less than 1480 bytes of traffic, we use complement 0 to expand it to 1480 bytes for storage.

Network traffic identification framework

Convolutional neural network architecture

The design process of the deep learning network model are proposed in this section. The original flow data is first input into the preprocessing module, and then output data that can be directly used by the convolutional neural network via four steps: header information processing [36], key information extraction, and data reprocessing. The preprocessed training data is then fed into the deep learning network training module [37], where the convolutional neural network model is trained through feature extraction, data simplification, category judgment, and feedback adjustment successively. Finally, the test data is fed into the test module, which contains the trained convolutional neural network model, and the system is evaluated and elaborated based on the classification results.

HexCNN-1D model structure design

The one-dimensional convolutional neural network (HexCNN-1D) workflow is based on the network traffic recognition method. The input data of the model are the hexadecimal data obtained after preprocessing. After training the model, the network traffic identification work is completed according to the different traffic categories.

To prevent overfitting, we added an attention mechanism and a batch normalization layer to the design of the HexCNN-1D model. Normalization returns an uneven distribution to a normalized distribution. This allows the processing data to be distributed into sensitive regions of the activation function, speeding up model training and preventing gradient disappearance.

The flow of the HexCNN-1D algorithm based on a convolutional neural network is shown in Fig. 1.

Fig. 1

HexCNN-1D Model Flow Chart

Batch normalization and attention mechanism addition

Considering the large amount and load of network traffic data to be processed, the traditional one-dimensional convolutional neural network model design cannot meet the lightweight requirement of identifying the types and categories of encrypted and malicious traffic with higher accuracy. Therefore, we add a normalized processing module and an attention mechanism module within our model.

Batch normalization addition

When designing the convolutional neural network model, the Batch Normalized (BN) module is considered as an addition to the normal convolutional neural network model [38]. The BN module can solve problems such as slow convergence rates and gradient saturation caused by internal covariate shift [39].

$${y}_{i}^{(b)}={BN({x}_{i})}^{(b)}=\upgamma \left(\frac{{x}_{i}^{(b)}-\mu \left({x}_{i}\right)}{\sqrt{\sigma {\left({x}_{i}\right)}^{2}+\epsilon }}\right)+\beta$$

(1)

${{x}_{i}}^{(b)}$ represents the value of the $i-th$ input node of this layer when the $b-th$ sample of the current batch is input, ${x}_{i}$ for $[{x}_{i}^{1}, {x}_{i}^{2}, {x}_{i}^{3},\dots ,{x}_{i}^{m}]$ a row vector, length of batch size m, $\mu$ and $\sigma$ for the mean and standard deviation, $\epsilon$ division by zero to prevent the introduction of a minimum quantity (negligible), $\beta$ and $\gamma$ for the shift and scale parameters.

Attention mechanism addition

Due to the uneven distribution of data, the model will pay more attention to sufficient data, which will affect the final classification effect. As mentioned in this paper [40], CBAM is a lightweight general module, that can be applied to any CNN model and plays a non-negligible role in the application of GAB and CAB [41]. GAB and CAB can be used to learn the recognition features, so as to better resolve the problem of low accuracy caused by uneven data distribution.

$${M}_{c\_a}=(ReLU(Conv2(GAP({M}_{G\_IN}))))\otimes {M}_{G-IN}, M\in {R}^{H\times W\times C}, {M}_{G-IN}\in {R}^{H\times W\times {C}^{^{\prime}}}, {C}^{^{\prime}}=C/2$$

(2)

The channel attention feature ${M}_{c\_a}$ is calculated in Formula 2, where $H$ denotes the height, $W$ represents the width, $C$ represents the number of channels, and $ReLU$ represents the use of ReLU activation function, $GAP$ represents the global average pooling, ${M}_{G-IN}$ denotes the use of 1 × 1 convolution layer to reduce the number of channels.

$${M}_{G-OUT}={M}_{c\_a}\otimes (ReLU(C\_G({M}_{c\_a})))$$

(3)

The number of channels required for each category is calculated by ${M}^{\mathrm{^{\prime}}},{M}^{\mathrm{^{\prime}}}\in {R}^{H\times W\times ck}$, where $c$ is the number of channels needed to identify each category, and $k$ is the number of classes. Half of the features are retained by ${M}^{"}({M}^{"}={M}^{\mathrm{^{\prime}}})$, and the Dropout function is removed to make a prediction with all the features.

Formula 3 calculates the output of GAB, namely the spatial attention feature map ${M}_{G-OUT}$, ${M}_{G-OUT}={M}_{G-IN}$. ${M}_{G-OUT}$ is used to store the subtle and different information of each network traffic category in the detailed network traffic data, which is used as the input to the subsequent CAB.

$${S}_{i}=\frac{1}{n}\sum_{j=1}^{n}GMP({m}_{ij}^{"}), i=\left\{\mathrm{1,2},3,\dots ,k\right\}, S=\{{S}_{1},{S}_{2},{S}_{3},\dots ,{S}_{k}\}$$

(4)

As can be observed in Formula 4, ${S}_{i}$ represents the degree of significant response to the feature mapping of each category, $GMP$ represents the global maximum pooling, ${m}_{ij}^{"}$ represents the JTH feature of class $i$ in ${M}^{"}$ and the score $S$ of each category of network traffic is calculated by averaging the sum of ${M}^{\mathrm{^{\prime}}\mathrm{^{\prime}}}$ maximum pooling.

$${M}_{i\_avg}^{\mathrm{^{\prime}}}=\frac{1}{n}\sum_{j=1}^{n}{m}_{ij}^{\mathrm{^{\prime}}}, i=\left\{\mathrm{1,2},3,\dots ,k\right\}$$

(5)

In Formula 5, ${M}_{i\_avg}^{\mathrm{^{\prime}}}$ represents the feature output mapping feature map of the class $i$, and ${m}_{ij}^{\mathrm{^{\prime}}}$ represents the reaction of the JTH feature of the class $i$ in ${M}^{\mathrm{^{\prime}}}$. The sum of the characteristic fractions of each class is calculated and averaged.

$${A}_{CAB}=\frac{1}{k}\sum_{i=1}^{k}{S}_{i}{M}_{i\_avg}^{\mathrm{^{\prime}}},{A}_{CAB}\in {R}^{H\times W\times 1}$$

(6)

In Formula 6, ${A}_{CAB}$ is to multiply and average the calculated scores of each class and the semantic features of the class. It helps to differentiate areas of DR Grading.

$${M}_{C-OUT}={M}_{C-IN}\otimes {A}_{CAB}$$

(7)

Finally, as shown in Formula 7, ${M}_{C-OUT}$ is obtained by multiplying CAB and category attention ${A}_{CAB}$, enabling the model to obtain more accurate classification of different network traffic categories.

Experimental data and index setting

In this section, the public network datasets ISCX and USTC are used for experiments. The testing ratio of the training set was set to 7:3, and the sample set used in each experiment was described in detail.

Experimental metrics settings

In this research, four classification indices were used in the experiment: Accuracy, Precision, Recall, and F1-score. TP denotes the positive sample correctly predicted by the model, FN denotes the positive sample incorrectly predicted by the model, FN denotes the negative sample incorrectly predicted by the model, and TN denotes the negative sample correctly predicted by the model.

We use the ablation experiment and the confusion matrix [42] to validate the detection of different data traffic categories and the experimental results. Ablation experiments are commonly used in neural networks to learn about the network by deleting part of the network and studying its performance. The confusion matrix's function is to group the expected and actual results of all categories into the same table based on category. In this table, we can clearly observe the number of accurate and inaccurate recognitions for each category.

Dataset category classification

The ISCX dataset contains traffic characteristics and raw traffic (in PCAP format). In our experiment, the experimental environment was divided into two categories (VPN and non-VPN), nine and eighteen.

The UTSC dataset uses the class 7 + 3 (seven non-malicious traffic and three malicious traffic) categories in the UTSC dataset to determine the model's ability to detect malicious traffic. We have 1,000 of each, for a total of 10,000 samples. The experiment went through 50–60 iterations.

Configuration and parameter Settings

For hardware and software configuration, we have used python3, PC version of Windows 11, Processor 12th Gen Intel(R) Core (TM) i5-12500H 2.50 GHz, running memory 16.0 GB.

We iteratively optimized the hyperparameters of the model and conducted a lot of model tuning mainly for batch processing [43], optimizer, loss function, normalization operation, etc., as shown in Table 3 below, the optimal parameter settings of the model are provided. The Adam optimizer is capable of updating the model parameters by calculating gradient optimization. Softmax loss function, etc.

Table 3 Optimal hyperparameter setting

Experimental results of network traffic identification

Compared with HexCNN-1D methods

The following are the experimental findings of the HexCNN-1D convolutional neural network model in two classifications, nine classifications, eighteen classifications, and malicious and non-malicious classifications:

The HexCNN-1D model developed in this paper uses two different exposed data sets, as shown in Fig. 2, and the accuracy indices of all tests were kept above 98%.

Fig. 2

HexCNN-1D Mode Experimental Results

As shown in Fig. 3, the above experimental results and data show that the HexCNN-1D model designed in this paper has a higher classification recognition accuracy and a more efficient classification effect.

Fig. 3

Experimental Results of HexCNN-1D Model

Therefore, we suggest that the combination of a convolutional neural network and network traffic recognition can significantly improve the accuracy of network traffic classification technology and can be more successfully applied to network traffic detection.

As shown in Table 4, the USTC-TFC data set shows that the HexCNN-1D model has more than 98% identification accuracy against malicious traffic such as Zeus, Virut, and Nsis-ay. This shows that the HexCNN-1D model established in this paper possesses the capability to detect malicious traffic. The packet length of malicious traffic is longer than that of regular traffic. The model we designed can extract valid data fields and accurately identify different types of malicious traffic with limited packet length.

Table 4 Malicious traffic identification by HexCNN-1D

The deep learning convolutional neural network classification model HexCNN-1D was trained to extract different label features. Four independent scenario tests were set up to collect experimental data of the HexCNN-1D model and compare it with the classical machine learning model. As can be observed in Table 5, the model proposed in this paper is superior to other network machine learning models in identifying VPN and non-VPN traffic. Compared to the traditional model (Deep Packet, C4.5), the accuracy of our model is improved by 14% to 28%. Compared to the common 1D-CNN model, the accuracy of encapsulating network traffic in both Non-VPN and VPN is increased by about 3 percentage points.

Table 5 Comparison with experimental results of different models

Ablation experiments

In order to evaluate the effectiveness of the model by adding normalized processing and attention mechanisms, we performed ablation experiments on HexCNN-1D. As shown in Table 6, the model is mainly processed by a one-dimensional convolutional neural network, followed by modules for normalization processing and attention mechanism.

Table 6 Comparison of ablation experiments

First, a single one-dimensional convolutional neural network was tested to calculate the Accuracy, Precision, Recall and F1-score of the model. Then, the accuracy of F1-score and other indicators of the model were increased by about 3% after the addition of normalized processing. Finally, CAB and GAB were added to the base model, and the overall index increased by about 2%, indicating that the attention module improved the efficiency of the model in identifying network traffic categories.

Confusion matrix validation experiment results

We used the confusion matrix shown in Fig. 4 to verify the experimental data and the classification accuracy of the experimental results.

Fig. 4

The Obfuscation Matrix Verifies the Results

The experimental results show that the HexCNN-1D classification model adopted in this paper has higher accuracy in four experimental scenarios, and has achieved excellent recognition results in the scenarios of encrypted traffic and malicious traffic identification.

Conclusion

In this paper, a convolutional neural network model is designed to study network traffic recognition. In the data preprocessing stage, the influence of redundant information is ignored. The data preprocessing method was coupled with the convolutional neural network model designed by HexCNN-1D. Our model identifies traditional traffic data and VPN encapsulated traffic with an accuracy of 99%. We found that in the detection of malicious network traffic, such as Zeus, Virut and Nsis-ay, the accuracy of network traffic identification reached more than 98%. In the future, we will investigate the robustness of these models and the performance migration of the models under different flow modes.