[Golang] Zap log in Golang

Posted on By nim No Comments on [Golang] Zap log in Golang

https://betterstack.com/community/guides/logging/go/zap/

Đầu tiên chúng ta sẽ bắt đầu với code demo đơn giản:

package main

import (
    "go.uber.org/zap"
)

func main() {
    logger := zap.Must(zap.NewProduction())

    defer logger.Sync()

    logger.Info("Hello from Zap logger!")
}

you must create a zap.Logger instance before you can begin to write logs. The NewProduction() method returns a Logger configured to log to the standard error in JSON format, and its minimum log level is set to INFO

và output bạn nhận được là:

{"level":"info","ts":1684092708.7246346,"caller":"zap/main.go:12","msg":"Hello from Zap logger!"}

Mình sẽ giải thích chút về output log trên:

  1. {"level":"info":
    • level: This indicates the severity level of the log message. Common levels include debug, info, warn, error, etc. In this case, info suggests that the message is informational.
    • This is useful for filtering logs. For example, in a production environment, you might only want to see warn and error level messages.
  2. "ts":1684092708.7246346:
    • ts: Stands for timestamp. It represents the time at which the log message was generated.
    • The number 1684092708.7246346 is a Unix timestamp in seconds with fractional part for millisecond precision. It denotes the specific time in seconds since the Unix epoch (January 1, 1970). This precise timestamp can be crucial for debugging, especially when looking at the sequence of events.
  3. "caller":"zap/main.go:12":
    • caller: This part of the log indicates where in the code the log message was generated.
    • "zap/main.go:12" means the log message was generated in the main.go file (presumably of the zap package or project) on line 12. This information is very helpful in larger projects for quickly locating the source of a log message.
  4. "msg":"Hello from Zap logger!":
    • msg: This is the actual log message.
    • The message "Hello from Zap logger!" is a human-readable string that provides information about what the log is about. In this case, it seems to be a simple test or demonstration message.

You can also utilize the NewDevelopment() preset to create a Logger that is more optimized for use in development environments. This means logging at the DEBUG level and using a more human-friendly format:
Bạn sẽ replace 1 chút:

logger := zap.Must(zap.NewDevelopment())

và output nó sẽ như thế này:

2023-05-14T20:42:39.137+0100    INFO    zap/main.go:12  Hello from Zap logger!

Setting up a global logger

Chúng ta sẽ luôn muốn sử dụng init fuction để create Zap Log.

package main

import (
    "go.uber.org/zap"
)

func init() {
    zap.ReplaceGlobals(zap.Must(zap.NewProduction()))
}

func main() {
    zap.L().Info("Hello from Zap!")
}

This method replaces the global logger accessible through zap.L() with a functional Logger instance so that you can use it directly just by importing the zap package into your file.

Examining Zap’s logging API

Zap provides two primary APIs for logging:
– The first is the low-level Logger type that provides a structured way to log messages
– The SugaredLogger type which represents a more laid-back approach to logging

the low-level Logger type

func main() {
    logger := zap.Must(zap.NewProduction())

    defer logger.Sync()

    logger.Info("User logged in",
        zap.String("username", "johndoe"),
        zap.Int("userid", 123456),
        zap.String("provider", "google"),
    )
}

output:

{"level":"info","ts":1684094903.7353888,"caller":"zap/main.go:17","msg":"User logged in","username":"johndoe","userid":123456,"provider":"google"}

The second higher-level API is the SugaredLogger

func main() {
    logger := zap.Must(zap.NewProduction())

    defer logger.Sync()

    sugar := logger.Sugar()

    sugar.Info("Hello from Zap logger!")
    sugar.Infoln(
        "Hello from Zap logger!",
    )
    sugar.Infof(
        "Hello from Zap logger! The time is %s",
        time.Now().Format("03:04 AM"),
    )

    sugar.Infow("User logged in",
        "username", "johndoe",
        "userid", 123456,
        zap.String("provider", "google"),
    )
}

output:

{"level":"info","ts":1684147807.960761,"caller":"zap/main.go:17","msg":"Hello from Zap logger!"}
{"level":"info","ts":1684147807.960845,"caller":"zap/main.go:18","msg":"Hello from Zap logger!"}
{"level":"info","ts":1684147807.960909,"caller":"zap/main.go:21","msg":"Hello from Zap logger! The time is 11:50 AM"}
{"level":"info","ts":1684148355.2692218,"caller":"zap/main.go:25","msg":"User logged in","username":"johndoe","userid":123456,"provider":"google"}

Structured Logger: sử dụng Zap’s structured logger. API dài dòng hơn nhưng cũng cấp strongly-typed, structured logging (cấu trúc). mỗi log entry (mỗi khi bạn khai báo log) là một message đã được cấu trúc với các trường được defined rõ ràng.

Pros:

  1. Structured Logging: The fields are explicitly defined, which makes it easier for log parsing and indexing tools to process and query logs.
  2. Performance: Typically faster than the Sugared logger because it avoids reflection and type conversions.
  3. Type Safety: Helps catch errors at compile time, as each field requires a specific type.

Cons:

  1. Verbosity: More verbose to write, as each field needs to be explicitly mentioned.
  2. Less Flexible: Doesn’t support formatting log messages with printf-style syntax.

Sugared Logger: sử dụng Zap’s Sugared Logger. Cách thì nó sẽ link động hơn, giảm sử dài dòng, đơn giản hơn, và friendly hơn với developer.

Pros:

  1. Ease of Use: More straightforward syntax, especially for developers familiar with standard Go loggers.
  2. Flexibility: Supports printf-style formatting, which can be more intuitive for constructing messages.
  3. Compatibility: Makes it easier to migrate existing code that uses standard Go logging to Zap.

Cons:

  1. Performance: Slightly slower than the structured logger due to the use of reflection and runtime type assertions.
  2. Less Structured: Printf-style and infow-style logs are less structured, which could be a downside for parsing and indexing.

Build logger

Mục tiêu của mình là sẽ build logger và chúng ta có thể dễ dàng ghi log ở bất cứ đâu trong project của chúng ta.

/*
Refer to: nimtechnology.com
Maintainers: Nim
*/
package logger

import (
    "compress/gzip"
    "fmt"
    "go.uber.org/zap"
    "go.uber.org/zap/zapcore"
    "gopkg.in/natefinch/lumberjack.v2"
    "io"
    "os"
    "strings"
    "sync"
    "time"
)

// Global logger instance
var Log *zap.Logger
var SugaredLog *zap.SugaredLogger

// Mutex for file lock to prevent concurrent file access
var fileLock sync.Mutex

// InitLogger initializes the logger with file rotation and timestamped logs.
func InitLogger() {
    // Get the log level from the environment variable
    logLevelStr := os.Getenv("LOG_LEVEL")
    if logLevelStr == "" {
        logLevelStr = "INFO" // Default to INFO if LOG_LEVEL is not set
    }

    // Convert the log level string to a zap log level
    var level zapcore.Level
    switch strings.ToUpper(logLevelStr) {
    case "DEBUG":
        level = zap.DebugLevel
    case "INFO":
        level = zap.InfoLevel
    case "WARN":
        level = zap.WarnLevel
    case "ERROR":
        level = zap.ErrorLevel
    default:
        level = zap.InfoLevel // Default to INFO if the level is not recognized
    }

    // Create a file writer with log rotation
    file := zapcore.AddSync(&lumberjack.Logger{
        Filename:   "logs/app.log",
        MaxSize:    5,    // Megabytes
        MaxBackups: 10,   // Max number of backups
        MaxAge:     14,   // Days
        Compress:   true, // Enable compression
    })

    // Set log level
    logLevel := zap.NewAtomicLevelAt(level)

    // Define encoder configuration
    encoderCfg := zap.NewProductionEncoderConfig()
    encoderCfg.TimeKey = "timestamp"
    encoderCfg.EncodeTime = zapcore.ISO8601TimeEncoder
    encoderCfg.CallerKey = "caller"

    // Create console and file encoders
    consoleEncoder := zapcore.NewConsoleEncoder(encoderCfg)
    fileEncoder := zapcore.NewJSONEncoder(encoderCfg)

    // Core setup for logging to both stdout and file
    core := zapcore.NewTee(
        zapcore.NewCore(consoleEncoder, zapcore.AddSync(os.Stdout), logLevel),
        zapcore.NewCore(fileEncoder, file, logLevel),
    )

    // Create a logger instance with AddCallerSkip(1) to skip the logger package stack frame
    Log = zap.New(core, zap.AddCaller(), zap.AddCallerSkip(1))
    SugaredLog = Log.Sugar() // Create a sugared logger for easier logging
}

// TimestampedLumberjackWriter wraps lumberjack.Logger and adds timestamp-based file rotation.
type TimestampedLumberjackWriter struct {
    *lumberjack.Logger
    baseFilename string
}

// NewTimestampedLumberjackWriter creates a new TimestampedLumberjackWriter.
func NewTimestampedLumberjackWriter(filename string, maxSize, maxBackups, maxAge int, compress bool) *TimestampedLumberjackWriter {
    return &TimestampedLumberjackWriter{
        Logger: &lumberjack.Logger{
            Filename:   filename,
            MaxSize:    maxSize,
            MaxBackups: maxBackups,
            MaxAge:     maxAge,
            Compress:   compress,
        },
        baseFilename: filename,
    }
}

// Write writes to the original log file and rotates it with a timestamp when it's closed.
func (t *TimestampedLumberjackWriter) Write(p []byte) (n int, err error) {
    fileLock.Lock() // Lock the file access
    defer fileLock.Unlock()

    n, err = t.Logger.Write(p)
    if err != nil {
        return n, err
    }

    // Create timestamped rotated filename
    timestamp := time.Now().Format("2006-01-02_15-04-05")
    rotatedFilename := fmt.Sprintf("%s.%s.gz", t.baseFilename, timestamp)

    // Attempt to compress and rename the log file with retry mechanism
    err = t.compressAndRename(rotatedFilename)
    return n, err
}

// compressAndRename handles the renaming and compressing of the log file.
func (t *TimestampedLumberjackWriter) compressAndRename(newFilename string) error {
    var err error
    for i := 0; i < 3; i++ {
        err = t.tryCompressAndRename(newFilename)
        if err == nil {
            return nil
        }
        time.Sleep(time.Second * 1) // Retry after a second
    }
    return err
}

// tryCompressAndRename performs the actual compression and renaming.
func (t *TimestampedLumberjackWriter) tryCompressAndRename(newFilename string) error {
    // Open the current log file
    currentFile, err := os.Open(t.baseFilename)
    if err != nil {
        return err
    }
    defer currentFile.Close()

    // Create a temporary file
    tmpFile, err := os.Create(newFilename + ".tmp")
    if err != nil {
        return err
    }
    defer tmpFile.Close()

    // Rename the current log file to temporary file
    err = os.Rename(t.baseFilename, tmpFile.Name())
    if err != nil {
        return err
    }

    // Compress the temporary file
    err = compressFile(tmpFile.Name())
    if err != nil {
        return err
    }

    // Finally, rename the temporary file to the desired final filename
    err = os.Rename(tmpFile.Name(), newFilename)
    return err
}

// compressFile compresses the log file to gzip format.
func compressFile(filename string) error {
    // Open the renamed log file
    file, err := os.Open(filename)
    if err != nil {
        return err
    }
    defer file.Close()

    // Create a gzip file with a .gz extension
    compressedFile, err := os.Create(filename + ".gz")
    if err != nil {
        return err
    }
    defer compressedFile.Close()

    // Compress the log file
    gzWriter := gzip.NewWriter(compressedFile)
    defer gzWriter.Close()

    // Copy the contents of the original file into the gzip file
    _, err = io.Copy(gzWriter, file)
    return err
}

// Wrapper functions for Infof, Debugf, Warnf, and Errorf

func Infof(format string, args ...interface{}) {
    SugaredLog.Infof(format, args...)
}

func Debugf(format string, args ...interface{}) {
    SugaredLog.Debugf(format, args...)
}

func Warnf(format string, args ...interface{}) {
    SugaredLog.Warnf(format, args...)
}

func Errorf(format string, args ...interface{}) {
    SugaredLog.Errorf(format, args...)
}

// Function to log with structured logging
func Info(msg string, fields ...zap.Field) {
    Log.Info(msg, fields...)
}

func Debug(msg string, fields ...zap.Field) {
    Log.Debug(msg, fields...)
}

func Warn(msg string, fields ...zap.Field) {
    Log.Warn(msg, fields...)
}

func Error(msg string, fields ...zap.Field) {
    Log.Error(msg, fields...)
}

// ZapWriter implements io.Writer, forwarding logs to a Zap logger.
type ZapWriter struct {
    logger *zap.Logger
    level  zapcore.Level
}

// NewZapWriter creates a new ZapWriter instance.
func NewZapWriter(logger *zap.Logger, level zapcore.Level) *ZapWriter {
    return &ZapWriter{logger: logger, level: level}
}

// Write satisfies io.Writer interface, forwarding output to the Zap logger.
func (z *ZapWriter) Write(p []byte) (n int, err error) {
    msg := strings.TrimSpace(string(p))
    switch z.level {
    case zapcore.DebugLevel:
        z.logger.Debug(msg)
    case zapcore.InfoLevel:
        z.logger.Info(msg)
    case zapcore.WarnLevel:
        z.logger.Warn(msg)
    case zapcore.ErrorLevel:
        z.logger.Error(msg)
    default:
        z.logger.Info(msg)
    }
    return len(p), nil
}

Tiếp đến ở hàm main thì bạn sẽ cần gọi InitLogger lên 1 lần

package main

import (
    "learn_zap_log/logger"
    "os"
)

func init() {
    // Set log level to DEBUG
    os.Setenv("LOG_LEVEL", "DEBUG")
    logger.InitLogger()
}

func main() {
    Hoge()
}

mình test việc gọi log trong file hope.go

package main

import (
    "go.uber.org/zap"
    "learn_zap_log/logger"
)

func Hoge() {
    url := "http://example.com"
    logger.Info("This is a function in some other package "+url, zap.String("key", "value"), zap.Int("count", 42))
    logger.Debug("This is a debug message")
    logger.Warn("This is a warning message")
    logger.Error("This is an error message")
}

Để sử dụng đc log debug thì bạn phải thêm biến môi trường là LOG_LEVEL = DEBUG
Nó tạo ra folder là logs và lưu file ở đó.

Golang

More Related Articles

[Golang/Swagger] Apply Swagger to describe the information API on Golang Golang
[Golang] Create Your Own CLI — With Golang Golang
[Golang] Using Repository Pattern to integrate the Database (Postgres) Golang
[aws-sdk-go] Integrate with AWS by golang Golang
[Golang] Writing file in Golang Golang
[Golang] Design config file or environment file with Golang Golang

Leave a Reply

Your email address will not be published. Required fields are marked *