preface

In the project development, the log system is an important component of the system module, through the program to record the run log, error log, can let us do a good control of the system running situation. At the same time, the collection of logs can not only be used to diagnose errors, because logs are also a large amount of data, and centralized analysis of such data can be of great value.

In the microservice system architecture, a system will be divided into many function modules, each module is responsible for different functions, and has higher requirements on the log system. The common solution is EFLK(ElasticSearch + Filebeat + LogStash + Kibana). As for our single application, because the code of the program is relatively concentrated, we mainly use handwritten log help class or use the form of third-party components to record the log information.

ASP.NET Core project

Storage address: github.com/Lanesra712/…

Step by Step

Why NLog and MongoDB

In ASP.NET Core, giant hardware provides an ILogger interface that makes it easy to export log information to the console. However, it is not convenient to view the log information in the console. We can implement this interface or directly use a third party framework to achieve log information to other storage media.

In the.net Framework era, most children will choose log4Net as their first choice of third-party logging Framework. However, since log4Net has not been updated for nearly 3 years, it is not considered. After a comprehensive comparison of several third-party log frameworks recommended in the official documents, WE finally choose NLog, which is currently used by a relatively large number of people. After all, if there are many users, it is easy to find information when encountering any problems.

Generally, log information is stored in TXT or log files. Although you can modify the log layout to make the log information readable, it is not convenient to view the log information in the presence of too much information. Writing log information to a relational database is not particularly appropriate because you need to log not only error messages but also program access logs at run time.

MongoDB as a document-type NoSQL database, compared with traditional relational database, NoSQL database has better scalability and can provide better performance. Therefore, I finally choose to record log information in MongoDB. Of course, the main reason is that I started to use MongoDB to store files uploaded by users in my work. In the process of searching for materials, I saw some cases that used MongoDB to store logs, and since VuCore is a learning project, I had to try more.

Install MongoDB (Windows)

Since this is the first time to use MongoDB, we need to install MongoDB Server in advance. I installed it directly on my development machine (Windows 10), so this is just to show how to install and configure MongoDB on Windows. I will demonstrate how to install and configure in Linux or Docker in a later article. After all, the project is eventually ready to deploy to Linux via Docker, and always playing on Windows is not appropriate drops.

First of all, open the MongoDB official website to obtain our MongoDB Community Download address, select the Server TAB and select the installation package according to our operating system to Download.

Double-click the downloaded MSI file to start the installation. Here I select Complete Installation. If you want to specify which components to install and where to install them, you can select Custom.

In previous MongoDB versions, if we wanted to use MongoDB Server as a Windows service, we had to configure it after installation, but starting with MongoDB 4.0, We can then directly configure and start our MongoDB as a Windows service during the installation, and when the installation is successful the MongoDB service will start automatically. Well, trust me, if you go online and do a search for MongoDB installations on Windows, you’ll find that 90% of the articles, because they were for previous versions of MongoDB, will require you to specify logging addresses, specify storage addresses, configure Windows services, and if you’re like me, Install MongoDB 4.0 or higher, do not need all of these, is not worth it.

Check Install MongoD as a Service and the MongoDB Service will be automatically started when the installation is complete. At the same time, we will not make any changes to the configuration items here.

Service Name: Specifies the Name of the created Windows Service. If an existing Service exists, change the Name

Data Directory: indicates the Directory where Data is stored

Log Directory: Directory for storing MongoDB Log logs

When we complete the installation, MongoDB service has been started, you can connect to your MongoDB Server, here I use Navicat to connect. For this service, you can also manage the service in Computer Management.

By default, when we install MongoDB, remote access is not allowed and there are no user permissions. And these, in our official use are all need to consider.

First, configure our MongoDB Server to allow remote access. Go to the mongod. CFG file (C: Program Files\MongoDB\Server\4.0\bin if you are using the default configuration) under the Program installation path and change the bindIp property to 0.0.0.0. After you restart the MongoDB service and ensure that port 27017 is accessible, you can remotely access the MongoDB service.

Open Navicat and connect to the installed MongoDB service.

Use admin to create an administrator account db.createuser ({user:"user name".pwd: "user password",
     roles: [ { role: "root", db: "admin"}]})Copy the code

Once we have created the administrator user, we can configure users and permissions for the database. GrapefruitVuCore right-click on the connection name and create a GrapefruitVuCore database, then switch to the GrapefruitVuCore database and create a GrapefruitVuCore user that can read and write. When all the users are created, close our MongoDB connection.

Use GrapefruitVuCore to create a admin user db.createuser ({user: GrapefruitVuCore)"grapefruit".pwd: "grapefruit",
     roles: [ { role: "readWrite", db: "GrapefruitVuCore"}]})Copy the code

Once the user has been created, we can modify the configuration file to enable permission control. In mongod.cfg, uncomment the security node and add authorization configuration. After modification, restart the service. At this point, MongoDB must be logged in by account password.

PS: the GrapefruitVuCore will not be showing under the right TAB but it will be real so I’m wondering what the problem is.

Read: Allows users to Read authorized databases

ReadWrite: allows users to read and write authorization databases

DbAdmin: allows users to perform administrative operations in authorized databases, such as index creation and deletion, viewing statistics, or accessing system.profile

UserAdmin: Allows users to write to the System. users collection and to create, delete, and manage users in the specified database

ClusterAdmin: Available only in the Admin database, it grants the management rights to all sharding and replication set-related functions.

ReadAnyDatabase: only available in the admin database, granting the user read permission for all databases

ReadWriteAnyDatabase: This parameter is available only in the admin database and is granted read and write permissions to all databases

UserAdminAnyDatabase: Only available in the admin database, granting the user the userAdmin permission for all databases

DbAdminAnyDatabase: only available in the admin database, giving the user the dbAdmin permission for all databases.

Root: available only in the admin database. Super account, super permission

3. Use NLog to record log information

After we have installed and configured MongoDB, we can use NLog to record our application logs. First, we need to add a reference to NLog for our project. Right-click estades.webAPI to open the Management Nuget package page or use the package manager console to select the default project estades.webapi. Add NLog, nlog.web. AspNetCore, and nlog.mongo.

Install-Package NLog
Install-Package NLog.Web.AspNetCore
Install-Package NLog.Mongo
Copy the code

ASP.NET

Once we’ve added the reference, add the NLog configuration file nlog.config under estades.webapi (all file names need to be lower case), right-click nlog.config, open the properties window, copy to the output directory, change it to newer, or always copy.

In the configuration file, the nlog node must be the root node of the XML file and contain three main child nodes: Extensions, Targets, and Rules.

Extensions: When you use more than just NLog as a basic DLL and use tools based on NLog extensions, you need to add the referenced assembly name under the Extensions node. Here, for example, I added the nlog.web. AspNetCore assembly to support NLog for ASP.NET Core, and the nlog. Mongo assembly to export logs to MongoDB.

The targets: The targets node contains the log information content and log information layout. For example, I output two files nlog-all-date.log and nlog-own-date.log by date to record all log information and customized log information respectively. Since we need to write log information to MongoDB, I also added a child node to set the data fields to be written to the MongoDB database.

Rules: The rules node associates the level of logs to be logged with the method of logging. In this case, I’m logging everything above Trace.

<?xml version="1.0" encoding="utf-8"? >
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      autoReload="true"
      internalLogLevel="info"
      internalLogFile="c:\Temp\GrapefruitVuCore\internal-nlog.txt">

  <! -- enable asp.net core and mongodb layout renderers -->
  <extensions>
    <add assembly="NLog.Web.AspNetCore"/>
    <add assembly="NLog.Mongo"/>
  </extensions>

  <! -- nlog: nlog to start and load config information -->
  <! --nlog-all: all logs -->
  <! --nlog-own: custom log information -->

  <! -- the targets to write to -->
  <targets>
    <! -- write logs to file -->
    <target xsi:type="File" name="allfile" fileName="c:\Temp\GrapefruitVuCore\nlog-all-${shortdate}.log"
            layout=${longDate}${newline} Level :${uppercase:${level}}${newline} Source :${logger}${newline} ${exception:format=tostring}${newline}==============================================================${newline}" />

    <! -- another file log, only own logs. Uses some ASP.NET core renderers -->
    <target xsi:type="File" name="ownFile-web" fileName="c:\Temp\GrapefruitVuCore\nlog-own-${shortdate}.log"
            layout=${longDate}${newline} Level :${uppercase:${level}}${newline} Source :${logger}${newline} ${exception:format=tostring}${newline}url: ${aspnet-request-url}${newline}action: ${aspnet-mvc-action}${newline}==============================================================${newline}" />

    <! -- write log to mongodb-->
    <target xsi:type="Mongo"
            name="mongo" databaseName="GrapefruitVuCore"
            collectionName="Logs"
            connectionString="mongodb://grapefruit:grapefruit@localhost:27017/GrapefruitVuCore"
            cappedCollectionSize="26214400">
      <property name="LongDate" layout="${longdate}" bsonType="DateTime" />
      <property name="Level" layout="${level}" />
      <property name="Logger" layout="${logger}"/>
      <property name="Message" layout="${message}" />
      <property name="Exception" layout="${exception:format=tostring}" />
      <property name="Url" layout="${aspnet-request-url}" />
      <property name="Action" layout="${aspnet-mvc-action}" />
      <property name="UserName" layout="${windows-identity}" />
    </target>

  </targets>

  <! -- rules to map from logger name to target -->
  <rules>
    <! --All logs, including from Microsoft-->
    <logger name="*" minlevel="Trace" writeTo="allfile" />

    <! --Skip non-critical Microsoft logs and so log only own logs-->
    <logger name="Microsoft.*" maxLevel="Info" final="true" />
    <! -- BlackHole without writeTo -->
    <logger name="*" minlevel="Trace" writeTo="ownFile-web" />

    <! --Add logs to mongodb-->
    <logger name="*" minlevel="Trace" writeTo="mongo"/>
  </rules>
</nlog>	
Copy the code

Public class Program {public static void Main(string[] args) {// Load log configuration file to catch all errors var logger = NLogBuilder.ConfigureNLog("nlog.config").GetCurrentClassLogger(); try { logger.Info("Init Log API Information"); CreateWebHostBuilder(args).Build().Run(); } catch (Exception ex) { logger.Error(ex, "Stop Log Information Because Of Exception"); } finally { LogManager.Shutdown(); } } public static IWebHostBuilder CreateWebHostBuilder(string[] args) => WebHost.CreateDefaultBuilder(args) .UseStartup<Startup>() .ConfigureLogging(logging => { logging.ClearProviders(); / / remove other registered log handler logging. SetMinimumLevel (Microsoft) Extensions) logging. The LogLevel. Trace); }).usenlog (); // Inject NLog service}Copy the code

In addition, the Logging configuration specified in appSettings. json overrides any calls to the SetMinimumLevel method. Therefore, you can remove the default attribute from the configuration file, or adjust it to suit your own needs.

{
    "Logging": {
        "LogLevel": {
            "Default": "Trace"."Microsoft": "Information"}}}Copy the code

conclusion

This chapter demonstrates how to install MongoDB Server on Windows and log information to MongoDB using NLog in an ASP.NET Core project. There are a lot of problems we may encounter when using these third-party open source frameworks. If you can’t solve them, the project Issue is a good place to do a lot of searching and there is a good chance you will find a solution.

Of the pit

Personal Profile: Born in 1996, born in a fourth-tier city in Anhui province, graduated from Top 10 million universities. .NET programmer, gunslinger, cat. It will begin in December 2016. NET programmer career, Microsoft. NET technology stalwart, aspired to be the cloud cat kid programming for Google the best. NET programmer. Personal blog: yuiter.com blog garden blog: www.cnblogs.com/danvic712