NICT Darknet Dataset 2022

Japanese Page (日本語)

tags: Darknet Dataset NICT

Dataset overview

This dataset is the darknet traffic data observed by NICTER.
It is a per-packet dataset made under the following conditions.

• Only TCP-SYN packets are released. (To analyze indiscriminate scanning attacks)
• A dataset in CSV format with the following data for each packet.

Contents of this site

1. How to use
   If you wish to use this dataset, please see [How to use](#How to use).
2. Pseudo data
   We present pseudo data in CSV format.
3. List of data periods and sensor IDs
   Please confirm the period and sensor ID of the data you want to utilize from this list.
   In order to ensure the reproducibility of our research, we have released the data for the period used in our research. We plan to continuously add and update the most recent data according to our research activities.

Differences from NICT Darknet Data Set 2019

In NICT Darknet Data Set 2019, statistical processing was done on each host, so its utility was limited. This dataset does not perform per-host statistical processing but only hashing, so it is considered to be more versatile in its use.

How to use

Pseudo data

This pseudo data is in CSV format.

• Before hashing

• After hashing

UNIXTIME,hash[ip.src.upper16],hash[ip.src.32],ip.dst.lower16,tcp.dport
1640962800.12,2fe1ec63c455bd46152926d283e91a8cc4a5fe4f471c27a56f825d046cdf8185,457d5c7b1a91d24d7747179ea793c009f509378781b7aaaa0c1748791b0108e0,100.100,23
1640962800.21,2d9e8afbdd75fd5a3be91f1fa290d4e43c90486a29519ceecd1ca5fd39dce22f,39825211c3134d68dd26708eb73fcad7c7fc3cf65b7a75e7fa8f9ab7c0c0c38e,100.101,2323
1640962800.21,1f5f57cbe46c479aef35f4dcb66d618c38d68fdc3739abe8b5e6fc0a5484c8fb,2ee37d765230eaa9f69a0508f0fc43589111b9e7c1a8ec26cd768d572defc1f6,101.100,80
1640962800.33,cf31089c853c78cfde5c57687cd3613288bd6ffc6c18dcf61a3a8cde7786d8bf,ea9eb9ad3e94e59103d4554332374c2fb19339ff5ba9e263e489edc6ce739f49,101.101,8080
1640962800.36,7f10d3eecd32bfb1c83b81238d42673b5c21b3c5533a6fa7ba7b5e2cf607430f,717aecfa766c462729db6b7443dbf928b61247142e3e575f9f4ba72a04420ff3,100.102,443

List of data periods and sensor IDs

Please check the period and sensor ID of the data you wish to use.

• To ensure the reproducibility of our research, we have released the data for the period used in our research.

  For this reason, in every period of data we publish, we include our references that conducted the research using the data.
  As mentioned in terms of use, please cite the paper [1] when you publish your work using this dataset. Also, please cite other references as necessary.

• We plan to continuously add and update the most recent data according to our research activities.

Data list

List of references

• [1] C. Han, J. Takeuchi, T. Takahashi, and D. Inoue, ‘‘Dark-TRACER: Early Detection Framework for Malware Activity Based on Anomalous Spatiotemporal Patterns,’’ IEEE ACCESS, 2022. [DOI] [PDF] [Slides]
• [2] C. Han, J. Shimamura, T. Takahashi, D. Inoue, M. Kawakita, J. Takeuchi, and K. Nakao, ‘‘Real-Time Detection of Malware Activities by Analyzing Darknet Traffic Using Graphical Lasso,’’ IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), 2019. [DOI] [PDF] [Slides]
• [3] C. Han, J. Shimamura, T. Takahashi, D. Inoue, J. Takeuchi, and K. Nakao, ‘‘Real-time Detection of Global Cyberthreat Based on Darknet by Estimating Anomalous Synchronization Using Graphical Lasso,’’ IEICE Transactions on Information and Systems, Vol.E103-D, No.10, pp.2113-2124, Oct. 2020. [DOI] [PDF]
• [4] C. Han, J. Takeuchi, T. Takahashi, and D. Inoue, ‘‘Automated Detection of Malware Activities Using Nonnegative Matrix Factorization,’’ IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), 2021. [DOI] [PDF] [Slides]
• [5] C. Han, A. Tanaka, and T. Takahashi, ‘‘Darknet Analysis-Based Early Detection Framework for Malware Activity: Issue and Potential Extension,’’ IEEE International Conference on Big Data (Workshop on Big Data for Cybersecurity), 2022. [DOI] [PDF] [Slides]
• [6] C. Han, A. Tanaka, J. Takeuchi, T. Takahashi, T. Morikawa, and T. Lin, ‘‘Towards Long-Term Continuous Tracing of Internet-Wide Scanning Campaigns Based on Darknet Analysis,’’ International Conference on Information Systems Security and Privacy (ICISSP), 2023. [DOI] [PDF] [Poster]
• [7] C. Han, A. Tanaka, T. Takahashi, S. Dadkhah, A. Ghorbani, and T. Lin, ‘‘Traceability Measurement Analysis of Sustained Internet-Wide Scanners via Darknet,’’ IEEE Conference on Dependable and Secure Computing (DSC), Nov 2024.

Person in charge

Cybersecurity Laboratory, Cybersecurity Research Institute, National Institute of Information and Communications Technology (NICT), Japan.

• Senior Manager, Takeshi Takahashi
• Researcher, Chansu Han
  • I am continuing my research using this darknet dataset. If you are interested, please refer to my biography below for darknet analysis papers.
  • https://hackmd.io/@gotchance/BJ7ACLZ0E

Contact for inquiries

For inquiries regarding the use of this dataset, please contact the following.

• ✉ csl-ai(at “@” symbol)ml(dot “.”)nict(dot “.”)go(dot “.”)jp

Acknowledgment

This effort was conducted under a contract of ‘‘MITIGATE’’ among ‘‘Research and Development for Expansion of Radio Wave Resources (JPJ000254),’’ which was supported by the Ministry of Internal Affairs and Communications, Japan.

Last updated on Nov 4, 2024
© NICT, Japan.
Chansu Han