preface
I wrote a log framework LogHelper before, which was packaged based on the open source library of Logger. At that time, the project itself had not many logs, so it could be used completely. Recently, I cooperated with other companies and gave feedback on a new project, saying that the use of the main functions of mobile phones would be affected in the case of a large number of logs. So THAT I did a deep reflection on the previous log behavior
Later, I consulted other developers in the development group. If you pursue performance, you can study wechat xlog, which is also the focus of this blog
What is the xlog
What is xlog this question I this is also in [Tencent Bugly dry share] wechat Mars high performance log module xlog got the answer
Simply put, it is the runtime log component based on C/C ++ with high reliability and high performance shared by Tencent team
How to use
If you know what it is, how to use it, open Github and find the official website Tencent/ Mars
Very simple to use
Download the library
Dependencies {compile 'com.tencent. Mars :mars-xlog:1.2.3'}Copy the codeuse
System.loadLibrary("c++_shared");
System.loadLibrary("marsxlog");
final String SDCARD = Environment.getExternalStorageDirectory().getAbsolutePath();
final String logPath = SDCARD + "/marssample/log";
// this is necessary, or may crash for SIGBUS
final String cachePath = this.getFilesDir() + "/xlog"
//init xlog
if (BuildConfig.DEBUG) {
Xlog.appenderOpen(Xlog.LEVEL_DEBUG, Xlog.AppenderModeAsync, cachePath, logPath, "MarsSample", 0, "");
Xlog.setConsoleLogOpen(true);
} else {
Xlog.appenderOpen(Xlog.LEVEL_INFO, Xlog.AppenderModeAsync, cachePath, logPath, "MarsSample", 0, "");
Xlog.setConsoleLogOpen(false);
}
Log.setLogImp(new Xlog());
Copy the codeOK implemented his function, don’t rejoice too soon, the subsequent problems will be big
Analyze the effects of each method
Knowing the simplest usage, I want to see what features it supports
AppenderOpen (appenderOpen
appenderOpen
appenderOpen(int level, int mode, String cacheDir, String logDir, String nameprefix, int cacheDays, String pubkey)
level
There’s nothing to say about the log level. XLog makes that very clear
public static final int LEVEL_ALL = 0;
public static final int LEVEL_VERBOSE = 0;
public static final int LEVEL_DEBUG = 1;
public static final int LEVEL_INFO = 2;
public static final int LEVEL_WARNING = 3;
public static final int LEVEL_ERROR = 4;
public static final int LEVEL_FATAL = 5;
public static final int LEVEL_NONE = 6;
Copy the codeNote In the DEBUG version, you are advised to enable console logs and set the log level to Verbose or DEBUG. In the Release version, you are advised to disable console logs and set the log level to Info. Details can be found in the access guide on the official website
This can also be done using the 👇 method
public static native void setLogLevel(int logLevel);
Copy the codeMode Write mode
-
public static final int AppednerModeAsync = 0; Asynchronous writes
-
public static final int AppednerModeSync = 1; Synchronous write
Synchronous write, can be regarded as real-time log, asynchronous is not. The Release version must use AppednerModeAsync, and the Debug version can work either way, but using AppednerModeSync may cause lag
This can also be set using the 👇 method
public static native void setAppenderMode(int mode);
Copy the codeCacheDir Sets the cache directory
Cache directory, when the writing will be written into this directory logDir, optional, don’t choose please show “”, if want to give, suggest to the application/data/data/packname/files/log directory.
A cache file with the suffix.mmap3 is generated in the directory,
LogDir Sets the file directory to be written to
The real log, with the suffix.xlog. Save logs to a separate directory. Do not put other files except log files in this directory; otherwise, the automatic log clearing function may delete them.
Nameprefix Sets the prefix of the log file name
For example, if the value is TEST, the generated file name is test_20170102.xlog.
cacheDays
CacheDays means the number of days after you move from the cache directory to the log directory. A non-0 parameter indicates how many days logs will be stored in the _cachedir directory.
The description here is a little more obscure, but when I set this parameter to non-0, I find that files that were set in the logDir directory appear in cacheDir.
🌰 Normally yes
File structure
- cacheDir
- log.mmap3
- logDir
- log_20200710.xlog
- log_20200711.xlog
Copy the codeIt looks like this. All under cacheDir
- cacheDir
- log.mmap3
- log_20200710.xlog
- log_20200711.xlog
- logDir
Copy the codePubkey Sets the encrypted pubkey
This involves the encryption and decryption of logs, which will be specially introduced below
setMaxFileSize
Setting file size
public static native void setMaxFileSize(long size);
Copy the codeIt represents the maximum file size, and it should be noted that the original default setting was one log file per day, as described clearly in appender.h
/* * By default, all logs will write to one file everyday. You can split logs to multi-file by changing max_file_size. * * @param _max_byte_size Max byte size of single log file, default is 0, meaning do not split. */
void appender_set_max_file_size(uint64_t _max_byte_size);
Copy the codeBy default, all logs are written to a file every day. You can split logs into multiple files by changing max_file_size. Maximum size of a log file in bytes. The default value is 0, indicating that the log file is not split
When the set file size is exceeded. The file becomes the following directory structure
- cacheDir
- log.mmap3
- logDir
- log_20200710.xlog
- log_20200710_1.xlog
- log_20200710_2.xlog
Copy the codeIn appender.cc there is the following logic:
static long __get_next_fileindex(const std::string& _fileprefix, const std::string& _fileext) {
...
return (filesize > sg_max_file_size) ? index + 1 : index;
Copy the codesetConsoleLogOpen
Sets whether to print logs on the console
public static native void setConsoleLogOpen(boolean isOpen);
Copy the codeSet whether logging is allowed on the console
setErrLogOpen
This method is useless, at first thought where inheritance problem, when viewing the source code found that he is an empty method, no application
Using it will cause exceptions in the program, so I have removed it in my own compilation
SetMaxAliveTime Sets the maximum retention time for a single file
public static native void setMaxAliveTime(long duration);
Copy the codeThe maximum retention time for a single file is in seconds. There are three points to note about this method.
- Must precede the appenderOpen method
- The minimum time is one day
- The default time is 10 days
This can be seen in appender.cc
static const long kMaxLogAliveTime = 10 * 24 * 60 * 60; // 10 days in second static const long kMinLogAliveTime = 24 * 60 * 60; // 1 days in second static long sg_max_alive_time = kMaxLogAliveTime; . void appender_set_max_alive_duration(long _max_time) { if (_max_time >= kMinLogAliveTime) { sg_max_alive_time = _max_time; }}Copy the codeThe default time is 10 days
appenderClose
An appenderClose method is described in the documentation to close the log when the program exits
In practice, however, onTerminate() of the Application class is valid only in the emulator, but not in the real machine.
If appenderClose is not triggered when the program exits, xlog will also write the log to the file at the next startup
So how do you trigger it?
It is recommended to trigger it as much as possible. For example, if the user double-clicks back to exit, you know that if it is killed in the background, there is really no way to refresh it. That’s ok.
appenderFlush
When the log write mode is asynchronous, calling this interface writes logs in memory to a file.
- IsSync: true: indicates synchronous flush. The value is returned only after the flush is complete.
- IsSync: false: indicates asynchronous flush, which is returned without waiting for the end of flush.
Encryption of log files
This piece is taken out separately to explain, because the previous use encountered a pit
First of all, the input parameter PUB_KEY, which is confused,
In Mars/blob/master/Mars/log/crypt/gen_key. The way this is to be able to get to the PUB_KEY py
Run the following
$ python gen_key.py
WARNING: Executing a script that is loading libcrypto in an unsafe way. This will fail in a future version of macOS. Set the LIBRESSL_REDIRECT_STUB_ABORT=1 in the environment to force this into an error.
save private key
471e607b1bb3760205f74a5e53d2764f795601e241ebc780c849e7fde1b4ce40
appender_open's parameter: 300330b09d9e771d6163bc53a4e23b188ac9b2f5c7150366835bce3a12b0c8d9c5ecb0b15274f12b2dffae7f4b11c3b3d340e0521e8690578f51813c 93190e1eCopy the codeThe private key above is saved by itself
Appender_open’s parameter: is the required PUB_KEY
Decryption of log files
Now that we know how to encrypt, how to decrypt it
Download pyelliptic1
You can see this in the Xlog encryption instructions
Pyelliptic1.5.7 needs to be downloaded and compiled otherwise the following command will fail
Direct decryption script
Xlog was kind enough to provide us with two scripts
Use decode_mars_nocrypt_log_file.py to unpack the unencrypted
python decode_mars_nocrypt_log_file [path]
Copy the codeFiles encrypted with decode_mars_crypt_log_file.py
Before using it, you need to add the
PRIV_KEY = "145aa7717bf9745b91e9569b80bbf1eedaa6cc6cd0e26317d810e35710f44cf8" PUB_KEY = "572d1e2710ae5fbca54c76a382fdd44050b3a675cb2bf39feebe85ef63d947aff0fa4943f1112e8b6af34bebebbaefa1a0aae055d9259b89a1858f7 cc9af9df1"Copy the codeChange it to the key you got from above otherwise you can’t extract it
python decode_mars_crypt_log_file.py ~/Desktop/log/log_20200710.xlog
Copy the codeDirectly generate a
- cacheDir
- log.mmap3
- logDir
- log_20200710.xlog
- log_20200710.xlog.log
Copy the codeYou can also customize the name
python decode_mars_crypt_log_file.py ~/Desktop/log/log_20200710.xlog ~/Desktop/log/1.log
Copy the code- cacheDir
- log.mmap3
- logDir
- log_20200710.xlog
- 1.log
Copy the codeChange the log format
Open our decompressed log to view
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Oct 14, 2019 ^ ^ ^ 20:27:59 ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 23 [17223172] [the 2020-07-24 + 0800 09:49:19] get mmap time: 3 MARS_URL: MARS_PATH: master MARS_REVISION: 85b19f92 MARS_BUILD_TIME: 2019-10-14 20:27:57 MARS_BUILD_JOB: log appender mode:0, use mmap:1 cache dir space info, capacity:57926635520 free:52452691968 available:52452691968 log dir space info, Available: available: available: available: available: available [I][2020-07-24 + 8.009:49:21.179][17223, 17223][,, [0] = = = = = = = = = = = = = = = = = = = = = = > 1 [I] [the 2020-07-24 + 8.0 09:49:21. 180] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 2 [I] [the 2020-07-24 + 8.0 09:49:21. 180] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 3 [I] [the 2020-07-24 + 8.0 09:49:21. 180] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 4 [I] [the 2020-07-24 + 8.0 09:49:21. 181] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 5 [I] [the 2020-07-24 + 8.0 09:49:21. 181] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 6 [I] [the 2020-07-24 + 8.0 09:49:21. 182] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 7 [I] [the 2020-07-24 + 8.0 09:49:21. 182] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 8 [I] [the 2020-07-24 + 8.0 09:49:21. 182] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 9 [I] [the 2020-07-24 + 8.0 09:49:21. 183] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 10 [I] [the 2020-07-24 + 8.0 09:49:21. 183] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 11 [I] [the 2020-07-24 + 8.0 09:49:21. 183] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 12 [I] [the 2020-07-24 + 8.0 09:49:21. 184] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 13 [I] [the 2020-07-24 + 8.0 09:49:21. 184] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 14 [I] [the 2020-07-24 + 8.0 09:49:21. 185] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 15 [I] [the 2020-07-24 + 8.0 09:49:21. 185] [17223, 17223] [TAG] [,, [0] = = = = = = = = = = = = = = = = = = = = = = > 16 [I] [the 2020-07-24 + 8.0 09:49:21. 185] [17223, 17223] [TAG] [, 0] [= = = = = = = = = = = = = = = = = = = = = = > 17Copy the codeI wipe tears apart from the information we need, there are so many miscellaneous miscellaneous 8 information, how to get rid of, and their definition of the format
Here we need to compile so ourselves, but xlog already provides us with good compilation code
The corresponding document is compiled locally
For the compilation part, just follow the documentation and notice that
- Always use the NDK-R20 instead of the latest version 21
- Be sure to use Python2.7 instead of Python3 for MAC
To reverse the file
First we go to this header file, which is useless for a logging framework
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Oct 14, 2019 ^ ^ ^ 20:27:59 ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 23 [17223172] [the 2020-07-24 + 0800 09:49:19] get mmap time: 3 MARS_URL: MARS_PATH: master MARS_REVISION: 85b19f92 MARS_BUILD_TIME: 2019-10-14 20:27:57 MARS_BUILD_JOB: log appender mode:0, use mmap:1 cache dir space info, capacity:57926635520 free:52452691968 available:52452691968 log dir space info, capacity:57926635520 free:52452691968 available:52452691968Copy the codeUnder Mars, go to appender.cc and remove the header file
Changing the Log Format
The default format is long
[I] [the 2020-07-24 + 8.0 09:49:21. 179] [17223, 17223] [TAG] [, 0] [= = = = = = = = = = = = = = = = = = = = = = > 1Copy the code[log level] [time] [pid, dar] [tag] [filename, strFuncName, line] [log content
It’s a structure that looks like this
It’s messy. All we want is the time, level, and content of the log
Find formater. Cc
Will the original
int ret = snprintf((char*)_log.PosPtr(), 1024."[%s][%s][%" PRIdMAX "%" PRIdMAX "%s][%s][%s, %s, %d][".// **CPPLINT SKIP**
_logbody ? levelStrings[_info->level] : levelStrings[kLevelFatal], temp_time,
_info->pid, _info->tid, _info->tid == _info->maintid ? "*" : "", _info->tag ? _info->tag : "",
filename, strFuncName, _info->line);
Copy the codeto
int ret = snprintf((char*)_log.PosPtr(), 1024."[%s][%s]".// **CPPLINT SKIP**
temp_time, _logbody ? levelStrings[_info->level] : levelStrings[kLevelFatal] );
Copy the codeWe have to do is
Then recompile and flip so into the project to see what it looks like now
[2020-07-24 +8.0 11:47:42.597][I]======================>9
Copy the codeOk, call it a day
Let’s just encapsulate it
Basically analyze and implement the functions we need, so this part of the simple encapsulation
The source Builder with the core attached can be viewed below
package com.allens.xlog
import android.content.Context
import com.tencent.mars.xlog.Log
import com.tencent.mars.xlog.Xlog
class Builder(context: Context) {
companion object {
// Log tag
var tag = "log_tag"
}
// Whether it is debug mode
private var debug = true
// Whether to print console logs
private var consoleLogOpen = true
// Is there a log file per day
private var oneFileEveryday = true
// The default location
private val defCachePath = context.getExternalFilesDir(null)? .path +"/mmap"
// Mmap location The location of the default cache
private var cachePath = defCachePath
// The actual location of the saved log
private var logPath = context.getExternalFilesDir(null)? .path +"/logDir"
For example, if the value is TEST, the generated file name is test_20170102.xlog
private var namePreFix = "log"
// Write file mode
private var model = LogModel.Async
// Maximum file size
// By default, all logs are written to a file every day. You can split logs into multiple files by changing max_file_size.
// Maximum size in bytes for a single log file. The default value is 0, indicating that the log file is not split
// The maximum file size cannot exceed 10M
private var maxFileSize = 0L
// Log level
You are advised to enable console logs in the DEBUG version and set the log level to Verbose or DEBUG. You are advised to disable console logs in the Release version and set the log level to Info.
private var logLevel = LogLevel.LEVEL_INFO
// Public key obtained from python gen_key.py
private var pubKey = ""
// Maximum retention time for a single file Minimum 1 day Default value 10 days
private var maxAliveTime = 10
// The number of cache days is usually set to 0. A non-0 parameter indicates how many days logs will be stored in the _cachedir directory.
// The cache date was moved from the cache directory to the log directory in a few days
private var cacheDays = 0
fun setCachePath(cachePath: String): Builder {
this.cachePath = cachePath
return this
}
fun setLogPath(logPath: String): Builder {
this.logPath = logPath
return this
}
fun setNamePreFix(namePreFix: String): Builder {
this.namePreFix = namePreFix
return this
}
fun setModel(model: LogModel): Builder {
this.model = model
return this
}
fun setPubKey(key: String): Builder {
this.pubKey = key
return this
}
// The original cache date means a few days later from the cache directory to log directory default 0
// Use [setMaxAliveTime] if you want to keep the file for several days
// When greater than 0, it is placed in the cache by default [cachePath]
fun setCacheDays(days: Int): Builder {
if (days < 0) {
this.cacheDays = 0
} else {
this.cacheDays = days
}
return this
}
fun setDebug(debug: Boolean): Builder {
this.debug = debug
return this
}
fun setLogLevel(level: LogLevel): Builder {
this.logLevel = level
return this
}
fun setConsoleLogOpen(consoleLogOpen: Boolean): Builder {
this.consoleLogOpen = consoleLogOpen
return this
}
fun setTag(logTag: String): Builder {
tag = logTag
return this
}
/** * [isOpen] true Sets a log file per day * false then setMaxFileSize takes effect */
fun setOneFileEveryday(isOpen: Boolean): Builder {
this.oneFileEveryday = isOpen
return this
}
fun setMaxFileSize(maxFileSize: Float): Builder {
when {
maxFileSize < 0- > {this.maxFileSize = 0L
}
maxFileSize > 10- > {this.maxFileSize = (10 * 1024 * 1024).toLong()
}
else- > {this.maxFileSize = (maxFileSize * 1024 * 1024).toLong()
}
}
return this
}
/** * [day] Sets the expiration time of a single file. By default, the expiration time of a single file is 10 days. The expiration time is calculated based on the current system time - the last modification time of a file
fun setMaxAliveTime(day: Int): Builder {
when {
day < 0- > {this.maxAliveTime = 0
}
day > 10- > {this.maxAliveTime = 10
}
else- > {this.maxAliveTime = day
}
}
return this
}
fun init(a) {
if(! debug) {// Enforce asynchrony if release is used
model = LogModel.Async
// The log level is Info
logLevel = LogLevel.LEVEL_INFO
}
if (cachePath.isEmpty()) {
/ / cachePath this parameter will pass, and private file directory under the data, for example/data/data/packagename/files/xlog, mmap files in this directory, if an empty string, the SIGBUS crash could occur.
cachePath = defCachePath
}
android.util.Log.i(tag, "Xlog=========================================>")
android.util.Log.i(
tag,
"info" + "\n"
+ "level:" + logLevel.level + "\n"
+ "model:" + model.model + "\n"
+ "cachePath:" + cachePath + "\n"
+ "logPath:" + logPath + "\n"
+ "namePreFix:" + namePreFix + "\n"
+ "cacheDays:" + cacheDays + "\n"
+ "pubKey:" + pubKey + "\n"
+ "consoleLogOpen:" + consoleLogOpen + "\n"
+ "maxFileSize:" + maxFileSize + "\n"
)
android.util.Log.i(tag, "Xlog=========================================<")
Xlog.setConsoleLogOpen(consoleLogOpen)
// One log file per day
if (oneFileEveryday) {
Xlog.setMaxFileSize(0)}else {
Xlog.setMaxFileSize(maxFileSize)
}
Xlog.setMaxAliveTime((maxAliveTime * 24 * 60 * 60).toLong())
Xlog.appenderOpen(
logLevel.level,
model.model,
cachePath,
logPath,
namePreFix,
cacheDays,
pubKey
)
Log.setLogImp(Xlog())
}
}
Copy the codeDownload the use
Step 1. Add the JitPack repository to your build file Add it in your root build.gradle at the end of repositories:
allprojects {
repositories {
...
maven { url 'https://www.jitpack.io' }
}
}
Copy the codeStep 2. Add the dependency
dependencies {
implementation 'com.github.JiangHaiYang01:XLogHelper:Tag'
}
Copy the codeAdd abiFilter
Android {compileSdkVersion 30 buildToolsVersion "30.0.1" defaultConfig {... ndk { abiFilter "armeabi-v7a" } } ... }Copy the codeCurrent Version
How to use
Initialization. It is recommended to place it in Application
XLogHelper.create(this) .setModel(LogModel.Async) .setTag("TAG") .setConsoleLogOpen(true) .setLogLevel(LogLevel.LEVEL_INFO) .setNamePreFix("log") .setPubKey("572d1e2710ae5fbca54c76a382fdd44050b3a675cb2bf39feebe85ef63d947aff0fa4943f1112e8b6af34bebebbaefa1a0aae055d925 9b89a1858f7cc9af9df1") .setMaxFileSize(1f) .setOneFileEveryday(true) .setCacheDays(0) .setMaxAliveTime(2) .init()Copy the codeuse
XLogHelper.i("======================> %s", i)
XLogHelper.e("======================> %s", i)
Copy the codegithub
All that writing and handing out XLogHelper
reference
Wechat Mars’s high-performance log module xlog is locally compiled