Monitoring System Architecture and Code Leak Prevention

Devops
1

Monitoring System Architecture and Code Leak Prevention

0x01 Summary

Code resources are the lifeblood of any organization. However, sensitive code is not something that should be spread outside the organization’s walls. Despite our best efforts, code leaks still occur due to various reasons. While it’s challenging to prevent code leaks entirely, we can certainly use technology to monitor and audit critical data, reducing the likelihood of resource leaks within a certain range. One popular method is to monitor GitHub for sensitive words. This article proposes an internal monitoring program to monitor SVN examples, focusing on the download behavior of relevant personnel within the system.

0x02 Key Resources and Roles

The process of data leakage involves key resources being downloaded from the repository to the local interior and then uploaded to GitHub. While listening to local developers is not ideal, we can monitor external GitHub repositories, which provide a relatively simple monitoring interface. This article focuses on internal warehouse monitoring and analysis, generating automated analysis reports and specific behavior of the system.

There are three resources involved in this process:

  1. Internal Warehouse: The code management system within the organization, which is not visible to outsiders, such as SVN.
  2. Warehouse Developers: The developers who work on the local warehouse.
  3. Github External Warehouse: The public GitHub repository, which is accessible to anyone.

0x03 Sensitive Resource Role Model Relationship

Production from code to resource consumption generally involves three roles:

  1. Code Submitter: The personnel who upload code works related to the code producer.
  2. Development Downloaders: The developers themselves who download code from the repository.
  3. Code Readers (Consumers): The local repository consumers who are related to the developers, as well as external GitHub users.

Our system adds a fourth role: Safety Management and Monitoring Personnel. We analyze resource leaks using two monitoring systems:

  1. Internal Warehouse Download Behavior Analysis System
  2. GitHub Sensitive Words Monitoring System

0x04 Monitoring Important Focus

The core focus of internal and external warehouse monitoring is:

Internal Warehouse Monitoring Focus:

  • Focus on key code downloads
  • Pay attention to abnormal excessive downloads
  • Special attention to user downloads

Internal Control System Results:

  • More downloads statistics table
  • Focusing resources are downloaded alarm

External Warehouse Monitoring Focus:

  • External warehouse monitoring is carried out by a keyword-associated monitoring of critical resources
  • This system is widely used by many companies

0x05 Warehouse Building Production Practices Internal Audit Analysis System

The internal audit system is a key component of the warehouse monitoring system. It involves collecting data, analyzing data, and followed by behavior analysis. The construction of the internal control system takes two steps:

  1. System Log Collection: SVN system log collection, log in traditional SVN server locally, needs to focus on the text log.
  2. Log Data Interface Technology: Automatic monitoring program, it is not a good way to interact directly read the text, if possible by calling the REST API, centralized monitoring and surveillance services can make policy instead.

We construct log data interface is divided into five steps:

  1. SVN text log → 2.rsync text server to a large capacity → 3. deploy server transmits the text to the syslog server nxlog → 4.syslog servers and local service data received text data into ES → 5. establish a data service gateways provide external REST API provides data query.

0x06 Listening Task Distribution Process

We use RPC to analyze user behavior. The construction of the above systems can be relatively easy to obtain log data through REST API. We focus on and achieve our monitoring strategy. Log data is automatically pulled for analysis through crontab and listening scheduling. We add a layer of task scheduling Wrapper application layer between real analysis and Cron, Cron just to execute Wrapper layer.

0x07 Summary

Automated tools can only monitor audit leak problem to some extent, but cannot stop a problem from the root. This article provides internal monitoring and external monitoring two kinds of programs:

  1. GitHub External Monitoring: We recommend a program: GitHub - freebuf-friends/x-patrol: github泄露扫描系统
  2. Internal Control: To build their own log system, we recommend articles to everyone’s engineering capabilities to build and implement the monitoring system is not difficult. ELK, SPLUNK, Graylog scene selection according to this system is the platform to build finished for multiple reuse, not only possible to analyze the behavior of SVN audit can analyze a variety of data behavior.

Related Articles