Taogen's Blog

Stay hungry stay foolish.

lobe-chat is a open-source project for build an AI client. It supports multiple AI providers, such as OpenAI, Claude 3, Gemini and more. It offers several useful features, including Local Large Language Model (LLM) support, Model Visual Recognition, TTS & STT Voice Conversation, Text to Image Generation, Plugin System (Function Calling), Agent Market (GPTs), Progressive Web App (PWA), Mobile Device Adaptation, and Custom Themes.

How to Deploy

Deploying with Docker

# Always pull the latest Docker image before running
docker pull lobehub/lobe-chat
docker run -d \
--name lobe-chat \
--restart always \
-p 3210:3210 \
-e OPENAI_API_KEY=sk-xxxx \
-e ACCESS_CODE=YOUR_PASSWORD \
lobehub/lobe-chat

Deploying to Vercel

You can also fork the lobe-chat project and deploy it to Vercel.

Setting Up lobe-chat

Required Settings

The API key is a required property that must be set.

If you set the OPENAI_API_KEY environment variable when you start the project, you can use the chatbot application directly. lobe-chat will not show an error or prompt you to set an API key. If you want to authenticate users, you can set the ACCESS_CODE environment variable.

If you don’t set the environment variables OPENAI_API_KEY and ACCESS_CODE when you start the project, lobe-chat will show an error on the web page and prompt you to set an API key. You can also set an API key in the settings page before using the chatbot.

Optional Settings

Set Default Agent

Model Settings

  • Model: Choose your preferred language model, such as GPT-4.

Set an API proxy

If you need to use the OpenAI service through a proxy, you can configure the proxy address using the OPENAI_PROXY_URL environment variable:

-e OPENAI_PROXY_URL=https://my-api-proxy.com/v1

If you want to use a localhost proxy

-e OPENAI_PROXY_URL=http://localhost:18080/v1 \
--network="host" \

or

# connect to proxy Docker container
-e OPENAI_PROXY_URL=http://{containerName}:{containerAppPort}/v1 \
--network {someNetwork} \

ChatGPT-Next-Web is an open-source project for building an AI chatbot client. This project is designed to be cross-platform, allowing it to be used on various operating systems. It currently can be used as a web or PWA application, or as a desktop application on Linux, Windows, or macOS. Additionally, it supports several AI providers, including OpenAI and Google AI.

How ChatGPT-Next-Web Works

ChatGPT-Next-Web manages your API keys locally in the browser. When you send a message in the chat box, ChatGPT-Next-Web will, based on your settings, send a request to the AI provider and render the response message.

How to Deploy

Deploying with Docker

# Always pull the latest Docker image before running
docker pull yidadaa/chatgpt-next-web
docker run -d \
--name chatgpt-next-web \
--restart always \
-p 3000:3000 \
yidadaa/chatgpt-next-web

Deploying to Vercel

You can also fork the ChatGPT-Next-Web project and deploy it to Vercel.

Setting Up ChatGPT-Next-Web

Click the settings button in the lower left corner to open the settings.

Required Settings

OpenAI API Key

Before using ChatGPT-Next-Web, you must set your OpenAI API Key in the Settings -> Custom Endpoint -> OpenAI API Key section.

Optional Settings

OpenAI Endpoint

If you have a self-deployed AI service API, you can set the value to something like http://localhost:18080.

Model

You can set your preferred model, such as gpt-4-0125-preview.

Others

Self-deployed AI services

You can use the copilot-gpt4-service to build a self-deployed AI service. To start an AI service, run the following command:

docker run -d \
--name copilot-gpt4-service \
--restart always \
-p 18080:8080 \
aaamoon/copilot-gpt4-service:latest

or

docker network create chatgpt

docker run -d \
--name copilot-gpt4-service \
--restart always \
-p 18080:8080 \
--network chatgpt \
aaamoon/copilot-gpt4-service:latest

OpenAI Proxy

openai-scf-proxy: Use Tencent Cloud Serverless to set up OpenAI proxy in one minute.

Gradle is a build automation tool for multi-language software development. It controls the development process in the tasks of compilation and packaging to testing, deployment, and publishing.

In this post, I will introduce the basic use of Gradle. This post is based on Gradle 8.6 and Kotlin DSL.

Initialize a Gradle Project

You can run the following commands to initialize a Java project with Gradle:

$ gradle init --use-defaults --type java-application

or

$ gradle init \
--type java-application \
--dsl kotlin \
--test-framework junit-jupiter \
--package my.project \
--project-name my-project \
--no-split-project \
--java-version 21

If you wan to create a Spring Boot application, you can use spring initializr.

Configurations

There are two configuration files in Gradle: build.gradle and settings.gradle. They are both important configuration files in a Gradle project, but they serve different purposes.

build.gradle is a script file that defines the configuration of a project. It’s written in the Groovy or Kotlin programming languages, and it specifies how tasks are executed, dependencies are managed, and artifacts are built. This file typically resides in the root directory of your project.

settings.gradle is focused on configuring the structure of a multi-project build and managing the relationships between different projects within it.

Plugins

You can add plugins to configuration file build.gradle.kts like this:

plugins {
java
id("org.springframework.boot") version "3.2.3"
id("io.spring.dependency-management") version "1.1.4"
}

Gradle core plugins:

  • java: Provides support for building any type of Java project.
  • application: Provides support for building JVM-based, runnable applications.

Spring Boot plugins:

  • org.springframework.boot: Spring Boot Gradle Plugin
  • io.spring.dependency-management: A Gradle plugin that provides Maven-like dependency management functionality. It will control the versions of your project’s direct and transitive dependencies.

More plugins:

Setting Properties

The following properties are the common properties for Java projects.

group = "com.example"
version = "0.0.1-SNAPSHOT"
java {
toolchain {
languageVersion = JavaLanguageVersion.of(21)
}
}
// or
java {
sourceCompatibility = "21" // JavaVersion.VERSION_21
targetCompatibility = "21"
}
application {
mainClass = "com.example.Main"
}

Repositories

A Repository is a source for 3rd party libraries.

repositories {
mavenCentral()
}

Declare dependencies

You can declare dependencies in build.gradle.kts like this

dependencies {
// Compile dependency
compileOnly('org.projectlombok:lombok:1.18.30')

// Implementation dependency
implementation("joda-time:joda-time:2.2")
implementation("org.springframework.boot:spring-boot-starter-web")

// Runtime dependency
runtimeOnly("com.mysql:mysql-connector-j")

// Test dependency
testImplementation("junit:junit:4.12")
testImplementation("org.springframework.boot:spring-boot-starter-test")
}

In Gradle, dependencies can be classified into several types based on where they come from and how they are managed. Here are the main dependency types:

  1. Compile Dependencies:
    1. These are dependencies required for compiling and building your project. They typically include libraries and frameworks that your code directly depends on to compile successfully.
    2. Dependencies declared with compile are visible to all modules, including downstream consumers. This means that If Module A has a compile dependency on a library, and Module B depends on Module A, then Module B also has access to that library transitively. However, this also exposes the implementation details of Module A to Module B, potentially causing coupling between modules. In Gradle 3.4 and later, compile is deprecated in favor of implementation.
  2. Implementation Dependencies:
    1. Introduced in Gradle 3.4, these dependencies are similar to compile dependencies but have a more restricted visibility.
    2. They are not exposed to downstream consumers of your library or module. This allows for better encapsulation and prevents leaking implementation details. This means that if Module A has an implementation dependency on a library, Module B, depending on Module A, does not have access to that library transitively. This enhances encapsulation and modularity by hiding implementation details of a module from its consumers. It allows for better dependency management and reduces coupling between modules in multi-module projects.
  3. Runtime Dependencies: Dependencies that are only required at runtime, not for compilation. They are needed to execute your application but not to build it.
  4. Test Dependencies: Dependencies required for testing your code. These include testing frameworks, libraries, and utilities used in unit tests, integration tests, or other testing scenarios.
  5. Optional Dependencies: Dependencies that are not strictly required for your project to function but are nice to have. Gradle does not include optional dependencies by default, but you can specify them if needed.

Tasks

tasks.withType<Test> {
useJUnitPlatform()
}

Run Tasks

To list all the available tasks in the project:

$ gradle tasks

Build Java

Before building a Java project, ensure that the java plugin is added to the configuration file build.gradle.kts.

plugins {
java
}

Running the following command to build the project

$ gradle build

Run Java main class

To run a Java project, ensure that the application plugin and the mainClass configuration are added to the configuration file build.gradle.kts. The application plugin makes code runnable.

plugins {
// Apply the application plugin to add support for building a CLI application in Java.
application
}

application {
// Define the main class for the application.
mainClass = "org.example.App"
}

Running the following command to run the main method of a Java project:

$ gradle run

Gradle Wrapper

The Gradle Wrapper is the preferred way of starting a Gradle build. It consists of a batch script for Windows and a shell script for OS X and Linux. These scripts allow you to run a Gradle build without requiring that Gradle be installed on your system.

The Wrapper is a script that invokes a declared version of Gradle, downloading it beforehand if necessary. As a result, developers can get up and running with a Gradle project quickly.

Gradle Wrapper files:

  • gradle/wrapper/gradle-wrapper.jar: The Wrapper JAR file containing code for downloading the Gradle distribution.
  • gradle/wrapper/gradle-wrapper.properties: A properties file responsible for configuring the Wrapper runtime behavior e.g. the Gradle version compatible with this version.
  • gradlew, gradlew.bat: A shell script and a Windows batch script for executing the build with the Wrapper.

If the project you are working on does not contain those Wrapper files, you can generate them.

$ gradle wrapper

Run tasks with gradlew:

$ ./gradlew tasks
$ ./gradlew build
$ ./gradlew run
$ ./gradlew test

References

[1] Building Java Projects with Gradle

[2] Part 1: Initializing the Project

[3] Build Init Plugin

[4] Gradle Wrapper Reference

[5] Spring Boot Gradle Plugin Reference Guide

Logback is a popular logging framework for Java applications, designed as a successor to the well-known Apache Log4j framework. It’s known for its flexibility, performance, and configurability. Logback is extensively used in enterprise-level Java applications for logging events and messages.

In this post, I will cover various aspects of using Logback with Spring Boot.

Dependencies

Before we can use Logback in a Spring Boot application, we need to add its library dependencies to the project.

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<version>{LATEST_VERSION}</version>
</dependency>

It contains ch.qos.logback:logback-classic and org.slf4j:slf4j-api

Logback Configuration Files

Spring Boot projects use logback-spring.xml or logback.xml in the resources directory as the Logback configuration file by default.

Priority of the Logback default configuration file: logback.xml > logback-spring.xml.

If you want to use a custom filename. You can specify the log configuration file path in application.xml or application.yml. For example:

logging:
config: classpath:my-logback.xml

Logback Basic Configurations

Property

You can define some properties that can be referenced in strings. Common properties: log message pattern, log file path, etc.

<configuration>
<property name="console.log.pattern"
value="%red(%d{yyyy-MM-dd HH:mm:ss}) %green([%thread]) %highlight(%-5level) %boldMagenta(%logger{36}:%line%n) - %msg%n"/>
<property name="file.log.pattern" value="%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"/>
<property name="file.log.dir" value="./logs"/>
<property name="file.log.filename" value="mylogs"/>

</configuration>

Appender

Appenders define the destination and formatting (optional) for log messages.

Types of Logback Appenders:

  • ConsoleAppender: Writes log messages to the console window (standard output or error).
  • FileAppender: Writes log messages to a specified file.
  • RollingFileAppender: Similar to FileAppender, but it creates new log files based on size or time intervals, preventing a single file from growing too large.
  • SocketAppender: Sends log messages over a network socket to a remote logging server.
  • SMTPAppender: Sends log messages as email notifications.

ConsoleAppender

<configuration>

<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${console.log.pattern}</pattern>
<charset>utf8</charset>
</encoder>
</appender>

</configuration>

FileAppender

<configuration>

<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>${file.log.dir}/${file.log.filename}.log</file>
<encoder>
<pattern>${file.log.pattern}</pattern>
</encoder>
</appender>

</configuration>

RollingFileAppender

<configuration>

<appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${file.log.dir}/${file.log.filename}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- Log file will roll over daily -->
<fileNamePattern>${file.log.pathPattern}</fileNamePattern>
<!-- Keep 30 days' worth of logs -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- Log messages greater than or equal to the level -->
<level>INFO</level>
</filter>
<encoder>
<pattern>${file.log.pattern}</pattern>
</encoder>
</appender>

</configuration>

RollingPolicy

A rollingPolicy is a component attached to specific appenders that dictates how and when log files are automatically managed, primarily focusing on file size and archiving. Its primary function is to prevent log files from becoming excessively large, improving manageability and performance.

Purpose:

  • Prevents large log files: By periodically rolling over (rotating) log files, you avoid single files growing too large, which can be cumbersome to manage and slow down access.
  • Archiving logs: Rolling policies can archive rolled-over log files, allowing you to retain historical logs for analysis or auditing purposes.

Functionality:

  • Triggers rollover: Based on the defined policy, the rollingPolicy determines when to create a new log file and potentially archive the existing one. Common triggers include exceeding a certain file size or reaching a specific time interval (e.g., daily, weekly).
  • Defines archive format: The policy can specify how archived log files are named and organized. This helps maintain a clear structure for historical logs.

Benefits of using rollingPolicy:

  • Manageability: Keeps log files at a manageable size, making them easier to handle and access.
  • Performance: Prevents performance issues associated with excessively large files.
  • Archiving: Allows you to retain historical logs for later use.

Common types of rollingPolicy in Logback:

  • SizeBasedTriggeringPolicy: Rolls over the log file when it reaches a specific size limit (e.g., 10 MB).
  • TimeBasedRollingPolicy: Rolls over the log file based on a time interval (e.g., daily, weekly, monthly).
  • SizeAndTimeBasedRollingPolicy: Combines size and time-based triggers, offering more control over rolling behavior.

TimeBasedRollingPolicy

<configuration>
<appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
...
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- Log file will roll over daily -->
<fileNamePattern>${file.log.dir}/${file.log.filename}-%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- Keep 30 days' worth of logs -->
<maxHistory>30</maxHistory>
</rollingPolicy>
...
</appender>
</configuration>

SizeAndTimeBasedRollingPolicy

<configuration>
<appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
...
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${file.log.dir}/${file.log.filename}-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
<!-- each archived file's size will be max 10MB -->
<maxFileSize>10MB</maxFileSize>
<!-- 30 days to keep -->
<maxHistory>30</maxHistory>
<!-- total size of all archive files, if total size > 100GB, it will delete old archived file -->
<totalSizeCap>100GB</totalSizeCap>
</rollingPolicy>
...
</appender>
</configuration>

Filter

A filter attached to an appender allows you to control which log events are ultimately written to the defined destination (file, console, etc.) by the appender.

Commonly used filters:

  • ThresholdFilter: This filter allows log events whose level is greater than or equal to the specified level to pass through. For example, if you set the threshold to INFO, then only log events with level INFO, WARN, ERROR, and FATAL will pass through.
  • LevelFilter: Similar to ThresholdFilter, but it allows more fine-grained control. You can specify both the level to match and whether to accept or deny log events at that level.

Filter only INFO level log messages.

<configuration>
<appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
...
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>INFO</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
...
</appender>
</configuration>

Filter level greater than INFO

<configuration>
<appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
...
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
...
</appender>
</configuration>

Logger

A logger in logback.xml represents a category or source for log messages within your application.

There are two types of logger tags in Logback: <root> and <logger>. They have hierarchical relationships. All <logger> are <root> child logger. Loggers can inherit their parent logger’s configurations. <root> represents the top level in the logger hierarchy which receives all package log messages. <logger> receives log messages from a specified package.

<configuration>

<root level="INFO">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</root>

<logger name="com.taogen" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</logger>

</configuration>
  • <root>: It receive all package log messages.
    • level="INFO": define the default logger level to INFO for all loggers.
    • <appender-ref>: Send messages to CONSOLE and ROLLING_FILE appender.
  • <logger>
    • name="com.taogen: It receive the com.taogen package log messages.
    • level="DEBUG": It overrides the logger level to DEBUG.
    • additivity="false": If the message has been sent to a appender by its parent logger, current logger will not send the message to the same appender again.
    • <appender-ref>: Send message to CONSOLE and ROLLING_FILE appender.

Using Logback

package com.taogen.commons.boot.mybatisplus;

import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;

@SpringBootTest(classes = AppTest.class)
@ExtendWith(SpringExtension.class)
@Slf4j
class LogTest {
private static Logger logger = LoggerFactory.getLogger(LogTest.class);

private static Logger customLogger = LoggerFactory.getLogger("my-custom-log");

@Test
void test1() {
log.debug("This is a debug message");
log.info("This is an info message");
log.warn("This is a warn message");
log.error("This is an error message");

logger.debug("This is a debug message");

customLogger.debug("This is a debug message");
customLogger.info("This is an info message");
customLogger.warn("This is a warn message");
customLogger.error("This is an error message");
}
}

@Slf4j is a Lombok annotation that automatically creates a private static final field named log of type org.slf4j.Logger. This log field is initialized with an instance of the SLF4J logger for the current class.

private static Logger log = LoggerFactory.getLogger(LogTest.class);

The commonly used Logback levels (in order of increasing severity):

  • TRACE: Captures the most detailed information.
  • DEBUG: general application events and progress.
  • INFO: general application events and progress.
  • WARN: potential problems that might not cause immediate failures.
  • ERROR: errors that prevent the program from functioning correctly.

Relationships between Logger object and <logger> in logback.xml

  • <logging> defined in logback.xml usually uses a package path as its name. Otherwise, use a custom name.
  • If you use Logback to print log messages in Java code, first, you need to pass a class or string to LoggerFactory.getLogger() method to get a Logger object, then call logger’s methods, such as debug().
  • If the Logger object is obtained through a class, Logback looks for <logger> from logback.xml using the object’s package or parent package path. If the Logger object is obtained through a string, Logback uses the string to find a custom <logger> from logback.xml.

More Configurations

Custom Loggers

You can create a custom logger by setting a name instead of using a package path as its name.

<configuration>

<logger name="my-custom-log" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE_CUSTOM"/>
</logger>

</configuration>

Output log messages:

2024-03-07 09:20:43 [main] INFO  my-custom-log - Hello!
2024-03-07 09:21:57 [main] INFO my-custom-log - Hello!
  • my-custom-log: logger name.

Note that if you use a custom logger, you can’t get class information from the log message.

Configurations for Different Environments

Using <springProfile>

<configuration>
...
. <!-- springProfile: 1) name="dev | test". 2) name="!prod" -->
<springProfile name="dev | test">
<root level="INFO">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</root>
<logger name="com.taogen.commons" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</logger>
</springProfile>
<springProfile name="prod">
<root level="INFO">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</root>
</springProfile>
</configuration>

Dynamically set the log configuration file path

You can dynamically set the log configuration file path in application.yml. Different spring boot environments use different log configuration files.

application.yml

logging:
config: classpath:logback-${spring.profiles.active}.xml

logback-dev.xml

<configuration>
...
<root level="INFO">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</root>
<logger name="com.taogen.commons" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</logger>
</configuration>

logback-prod.xml

<configuration>
...
<root level="INFO">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE"/>
</root>
</configuration>

A Complete Example

Goals

  • Properties
    • log patterns, log directory and log filename.
  • Appenders
    • Colorful log pattern for console appender.
    • Console and RollingFile appenders.
    • Time based rolling policy. Roll over daily, Keep 30 days’ worth of logs.
    • Filter log messages in appenders. Separate INFO, ERROR log messages.
  • Loggers
    • Setting loggers of the root and the base package of project.
    • Using custom loggers.
    • Support multiple spring boot environments.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>
<configuration>
<!-- Define properties. You can use these properties in appender configurations.-->
<property name="console.log.pattern"
value="%red(%d{yyyy-MM-dd HH:mm:ss}) %green([%thread]) %highlight(%-5level) %boldMagenta(%logger{36}:%line%n) - %msg%n"/>
<property name="file.log.pattern" value="%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"/>
<property name="file.log.dir" value="./logs"/>
<property name="file.log.filename" value="mylogs"/>

<!-- Define the CONSOLE appender. 1) log pattern. 2) log file path. -->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${console.log.pattern}</pattern>
<charset>utf8</charset>
</encoder>
</appender>

<!-- RollingFileAppender: Adds the capability to perform log file rotation. You can define a rolling policy, specifying criteria such as time-based or size-based rollover. -->
<appender name="ROLLING_FILE_INFO" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${file.log.dir}/${file.log.filename}-info.log</file>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- Log file will roll over daily -->
<fileNamePattern>${file.log.dir}/${file.log.filename}-info-%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- Keep 30 days' worth of logs -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${file.log.pattern}</pattern>
</encoder>
</appender>

<appender name="ROLLING_FILE_ERROR" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${file.log.dir}/${file.log.filename}-error.log</file>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>ERROR</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- Log file will roll over daily -->
<fileNamePattern>${file.log.dir}/${file.log.filename}-error-%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- Keep 30 days' worth of logs -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${file.log.pattern}</pattern>
</encoder>
</appender>

<appender name="ROLLING_FILE_CUSTOM" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${file.log.dir}/custom-logs.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- Log file will roll over daily -->
<fileNamePattern>${file.log.dir}/custom-logs-%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- Keep 30 days' worth of logs -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${file.log.pattern}</pattern>
</encoder>
</appender>

<!-- Define root logger. 1) Set the default level for all loggers. 2) Set which appenders to use. -->
<root level="INFO">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE_INFO"/>
<appender-ref ref="ROLLING_FILE_ERROR"/>
</root>
<!-- custom logger -->
<logger name="my-custom-log" level="INFO" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE_CUSTOM"/>
</logger>
<!-- springProfile: 1) name="dev | test". 2) name="!prod" -->
<springProfile name="dev | test">
<!-- Define loggers. Set log level for specific packages. -->
<logger name="com.taogen.commons" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING_FILE_INFO"/>
<appender-ref ref="ROLLING_FILE_ERROR"/>
</logger>
</springProfile>
</configuration>

Principles

  • Oriented toward novices. It is assumed that most of the readers of the document are novices. This way you will write more understandable, in-depth, detailed, and readable.

Structure

  • Overall logic: what, why, how, when, where.
  • Try to break out into detailed subdirectories. You can quickly locate what you want to see.

Details

  • The steps should be clear. Label steps 1, 2, and 3.
  • Try to add links to nouns that you can give links to. E.g. official website, explanation of specialized terms.
  • The code field is to be marked with a code, e.g. code.
  • Use tables as much as possible for structured information.
  • Try to use pictures where you can illustrate to make a clearer and more visual illustration. Don’t mind the hassle. It is more visual and easier to read. For example: UML, flow chart.
  • Give a link to the reference content at the end.

Others

  • After writing, read it through at least once. Timely detection and revision of some statement errors, incoherence; inaccuracy and lack of clarity of expression, and so on.

Write Statistical SQLs

Before write the code, you can write statistical SQLs first. Because the core of statistical APIs are SQLs. As the saying goes, “first, solve the problem, then, write the code”.

Define Parameter and Response VOs

VO (value object) is typically used for data transfer between business layers and only contains data.

Parameter VOs

@Data
public class SomeStatParam {
private Date beginTime;
private Date endTime;
private Integer type;
private Integer userId;
...
}

Response VOs

@Data
public class someStatResult {
...
}

Write the APIs

@RestController
@RequestMapping(value = "/moduleName")
public class SomeStatController {
@GetMapping(value = "/getSomeStat")
public ResponseEntity<SomeStatResult> getSomeStat(SomeStatParam someStatParam) {
SomeStatResult data = someStatService.getSomeStat(someStatParam);
return ResponseEntity.ok(data);
}
}

Test APIs in Your Local Environment

Before integrating third-party APIs in your web application, it’s better to test the APIs with an API tool like Postman to ensure the APIs work properly in your local environment. It can also let you get familiar with the APIs.

Add API Configurations to Your Project

There are some common configurations of third-party APIs you need to add to your project. For example, authorization information (appId, appSecret), API URL prefix, and so on.

Add Configurations

Add the configurations to your spring boot configuration file application.yaml:

thirdParty:
thirdParty1:
appId:
appSecret:
apiUrlPrefix:
thirdParty2:
...

Define the Configuration Bean

Add a spring bean to receive the configurations:

YourThirdPartyConfig.java

@Configuration
@EnableConfigurationProperties
@ConfigurationProperties(prefix = "thirdParty.yourThirdPartyName")
@Data
public class YourThirdPartyConfig {
protected String appId;
protected String appSecret;
protected String apiPrefix;
}

Access Your Configurations

Access the configurations by @Autowired it

@Autowired
private YourThirdPartyConfig yourThirdPartyConfig;
String appId = yourThirdPartyConfig.appId;
String appSecret = yourThirdPartyConfig.appSecret;

Write Code

Define the API Interfaces

Here we use the example of the social login API service to demonstrate.

public interface SocialLoginService {
JSONObject func1(JSONObject params);
JSONObject func1(JSONObject params);
}

Why use JSONObject as parameters and return types of methods of the interface?

Different third-party API service platforms have different data structures. The parameters and return types of methods need to be extensible.

Define Constants

Keys for API parameters

public class Auth0Key {
public static final String CODE = "code";
public static final String MSG = "msg";
public static final String USER_NAME = "user_name";
}

API request URIs

public class Auth0Url {
public static final String LOGIN = "/login";
public static final String GET_USER_INFO = "/user/getInfo";
}

Write the API Implementing Class

@Service
public class Auth0SocialLoginService implements SocialLoginService {
@Autowired
private Auth0Config auth0Config;

public JSONObject func1(JSONObject params) {
...
}
public JSONObject func2(JSONObject params) {
...
}
}

Common third-party APIs and their class names

API Service Interface Name Implementation Name
Cloud Object Storage COSService AwsCOSService
AzureCOSService
Push Notification APIs PushNotificationService OneSignalPushService
Social Media Login APIs SocialLoginService Auth0SocialLoginService
FirebaseSocialLoginService

Write Parameter Builders and Result Handlers

Parameter builders

public interface SocialLoginParamBuilder {
JSONObject getFunc1Param(Xxx entity);
JSONObject getFunc2Param(Xxx entity);
}
@Component
public class Auth0SocialLoginParamBuilder {
JSONObject getFunc1Param(Xxx entity) {...}
JSONObject getFunc2Param(Xxx entity) {...}
}

Result handlers

public interface SocialLoginResultHandler {
Vo handleFunc1Result(JSONObject result);
Vo handleFunc2Result(JSONObject result);
}
@Component
public class Auth0SocialLoginResultHandler {
Vo handleFunc1Result(JSONObject result) {...}
Vo handleFunc2Result(JSONObject result) {...}
}

Use Your Interface

@Autowired
@Qualifier("auth0SocialLoginService")
private SocialLoginService socialLoginService;
// or
@Autowired
private SocialLoginService auth0SocialLoginService; // The SocialLoginService object name must same as the implementing class name and the first character should be lowercase.

@Autowired
@Qualifier("auth0SocialLoginParamBuilder")
private SocialLoginParamBuilder socialLoginParamBuilder;

@Autowired
@Qualifier("auth0SocialLoginResultHandler")
private SocialLoginResultHandler socialLoginResultHandler;
JSONObject params = auth0SocialLoginParamBuilder.getFunc1Param(entity);
JSONObject result = auth0SocialLoginService.func1(params);
Vo vo = auth0SocialLoginResultHandler.handleFunc1Result(result);

This post will cover how to create a static website using VitePress.

Before using VitePress you must install Node.js v18 or higher.

Initialize VitePress Project

Add vitepress dependency

$ mkdir my-site
$ cd my-site

# add vitepress to devDependencies
$ npm add -D vitepress
# or
$ yarn add -D vitepress

VitePress is used in the development process as a build tool. It converts your Markdown files to HTML files. You don’t need VitePress in runtime.

Scaffold a basic project

$ npx vitepress init

npx can run your installed package directly, you don’t need to add any npm script to your package.json. You also can do npx package_command by using npm run your_script.

You need to set some basic configuration for your website.

  • Setting the VitePress directory. ./ means using the root directory.
  • Setting your site title.
  • Setting your site description.
  • Other configurations just use the default settings.
┌  Welcome to VitePress!

◇ Where should VitePress initialize the config?
│ ./

◇ Site title:
│ My Awesome Project

◇ Site description:
│ A VitePress Site

◇ Theme:
│ Default Theme

◇ Use TypeScript for config and theme files?
│ Yes

◇ Add VitePress npm scripts to package.json?
│ Yes

└ Done! Now run npm run docs:dev and start writing.

Running the project

Start a local server

npm run docs:dev
# or
yarn docs:dev
# or
npx vitepress dev

Visiting http://localhost:5173 to access the website

Git Configurations

Initialize the git repository

$ cd my-site
$ git init .

Config gitignore

Add the .vitepress/cache directory to .gitignore

.vitepress/cache

Result .gitignore

.idea

# macOS
.DS_Store

# vitepress
.vitepress/cache

# Dependency directories
node_modules/

# build output
dist

VitePress Configurations

Site Config

The site configuration file is .vitepress/config.mts

export default defineConfig({

srcDir: 'src',
srcExclude: [
'someDir/**',
'someFile',
],
// Whether to get the last updated timestamp for each page using Git.
lastUpdated: true,
head: [
['link', {rel: 'shortcut icon', type: "image/jpeg", href: '/logo.jpeg'}],
// These two are what you want to use by default
['link', {rel: 'apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "72x72", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "114x114", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "144x144", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon-precomposed', type: "image/jpeg", href: '/logo.jpeg'}],
// This one works for anything below iOS 4.2
['link', {rel: 'apple-touch-icon-precomposed apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
],

})
$ mkdir src/ && mv index.md src/

Files

  • src/public/logo.jepg

srcDir

Move the home page index.md to src/index.md

$ mkdir src/
$ mv index.md src/
export default defineConfig({

srcDir: 'src',

})

srcExclude

Optional config. If you need to add excluded directories and files.

export default defineConfig({

srcExclude: [
'someDir/**',
'someFile',
],

})

For example:

  • docs/**,
  • index-en.md

lastUpdated

export default defineConfig({

// Whether to get the last updated timestamp for each page using Git.
lastUpdated: true,

})

Shortcut icon

export default defineConfig({

head: [
['link', {rel: 'shortcut icon', type: "image/jpeg", href: '/logo.jpeg'}],
// These two are what you want to use by default
['link', {rel: 'apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "72x72", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "114x114", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon', type: "image/jpeg", sizes: "144x144", href: '/logo.jpeg'}],
['link', {rel: 'apple-touch-icon-precomposed', type: "image/jpeg", href: '/logo.jpeg'}],
// This one works for anything below iOS 4.2
['link', {rel: 'apple-touch-icon-precomposed apple-touch-icon', type: "image/jpeg", href: '/logo.jpeg'}],
],

})

src/public/logo.jepg

Theme Config

The theme configuration file is also .vietpress/config.mts

import {type DefaultTheme, defineConfig} from 'vitepress'

export default defineConfig({
themeConfig: {
nav: nav(),
sidebar: {
'/example/': sidebarExample()
},

logo: {src: '/your-logo.png', width: 24, height: 24},
search: {
provider: 'local'
},
outline: {
level: "deep"
},
}
})

function nav(): DefaultTheme.NavItem[] {
return [
{text: 'Home', link: '/'},
{text: 'Example', link: '/example/'},
];
}

function sidebarExample(): DefaultTheme.SidebarItem[] {
return [
{text: 'Examples', link: '/examples.md'},
]
}

Files

  • src/public/your-logo.png
export default defineConfig({
themeConfig: {

logo: {src: '/your-logo.png', width: 24, height: 24},

}
})

src/public/your-logo.png

export default defineConfig({
themeConfig: {

search: {
provider: 'local'
},

}
})

Outline

export default defineConfig({
themeConfig: {

outline: {
level: "deep"
},

}
})

Home Page Config

Config home page in src/index.md

Image

hero:
image:
src: your-home-image.png
alt: your-home-image

src/public/your-home-image.png

Feature Icons

features:
- icon: 🚀
title:
details:

Other Config

Favicon

Put your website icon to src/public/favicon.ico

Deployment

Create a repository on GitHub

Create button on the right top bar of GitHub -> New repository

Enter your repository name. For example, my-site

Click the “Create Repository” button.

Commit your local repository to GitHub

git add .
git commit -m '🎉feat: First commit'
git remote add origin git@github.com:{your_username}/{repo_name}.git
git branch -M main
git push -u origin main

Deploy your project on Vercel

Go to Vercel website.

Click the “Add New” button -> Project -> Import Git Repository

Add permission to access the new GitHub repository to Vercel

After finishing the permission configuration, you can find the new GitHub repository on Vercel.

Select the repository. And click “Import”

Override “Build and Output Settings”

  • Build Command: npm run docs:build or yarn docs:build
  • Output Directory: .vitepress/dist

Click “Deploy”

After the deployment process is finished, you can visit your website by Vercel provided URL.

Common text processing commands on Linux

  • grep
  • sed
  • awk
  • tr
  • sort
  • wc

Filter

Filter lines

echo -e "Foo\nBar" | grep "Fo"

Insert

Insert lines

Insert to the specific line

echo -e 'Foo\nBar' | sed '2i\the new line\' # Foo\nthe new line\nBar

Add line to beginning and end

echo -e 'Foo\nBar' | sed '1 i\First line'
echo -e 'Foo\nBar' | sed '$aEnd line'

Insert lines after match pattern

echo -e 'Foo\nBar' | sed '/Foo/a NewLine1\nNewLine2'
echo -e 'Foo\nBar' | sed '/Foo/r add.txt'

Insert text to the beginning and end

Insert text to the beginning of each line

echo 'foo' | sed 's/^/BeginText/'

Insert text to the end of each line

echo 'foo' | sed 's/$/EndText/'

Insert text to the begining and end of each line

echo 'foo' | sed 's/^/BeginText/' | sed 's/$/EndText/'

Insert a new line to the end of each line

echo -e 'Foo\nBar'| sed 's/$/\r\n/'

Replace

Replace first

echo "old names, old books" | sed 's/old/new/'
# or
echo "old names, old books" | sed '0,/old/{s/old/new/}'

Replace all

echo "old names, old books" | sed 's/old/new/g'

Remove

Remove matched lines

echo -e "Foo\nBar" | sed '/Foo/d'

Remove empty line

echo -e "Foo\n \nBar" | sed '/^\s*$/d'
# or
echo -e "Foo\n \nBar" | sed '/^[[:space:]]*$/d'

Remove comment /**/ or //

# reomve lines start with / or *
sed '/^ *[*/]/d'

Remove n lines after a pattern

# including the line with the pattern
echo -e "Line1\nLine2\nLine3\nLine4" | sed '/Line1/,+2d' # Line4

# excluding the line with the pattern
echo -e "Line1\nLine2\nLine3\nLine4" | sed '/Line1/{n;N;d}' # Line1\nLine4

Remove all lines between two patterns

# including the line with the pattern
sed '/pattern1/,/pattern2/d;'
echo -e "Foo\nAAA\nBBB\nBar\nCCC" | sed '/Foo/,/Bar/d' # CCC

# excluding the line with the pattern
sed '/pattern1/,/pattern2/{//!d;};'
echo -e "Foo\nAAA\nBBB\nBar\nCCC" | sed '/Foo/,/Bar/{//!d;}' # Foo\nBar\nCCC

Find String

Find String by Pattern

echo -e 'Hello Java developer!\nHello Web developer!' | sed 's/Hello \(.*\) developer!/\1/'

Join

Join lines

echo -e "Foo\nBar" | tr '\n' ' '

Split

Split to multiple lines

echo "Foo Bar" | tr '[:space:]' '[\n*]'

Deduplication

# sort and deduplication
echo -e "1\n3\n2\n1" | sort -u

Sort

echo -e "1\n3\n2\n1" | sort

Count

Count lines

echo -e "1\n3\n2\n1" | wc -l # 4

Count matched lines

echo -e "1\n3\n2\n1" | grep -c "1" # 2

Count matched number of string

echo "hello world" | grep -o -i "o" | wc -l # 2

Format

To Upper/Lower Case

# to upper case
echo "hello WORLD" | tr a-z A-Z
# to lower case
echo "hello WORLD" | tr A-Z a-z

Format JSON string

echo '{"name":"Jack","age":18}' | jq .
echo '{"name":"Jack","age":18}' | jq .name

Crypto

Encode

Base64

Base64 Encode

printf 'hello' | base64
# or
echo -n 'hello' | base64
  • -n: do not output the trailing newline

Base64 Decode

printf 'hello' | base64 | base64 -d

URL encode

URL encode

printf '你好' | jq -sRr @uri
# or
echo -n '你好' | jq -sRr @uri
  • -n: do not output the trailing newline

Hash

md5

md5 -r /path/to/file
printf 'hello' | md5sum

echo -n 'hello' | md5sum
# or
echo -n 'hello' | md5
  • -n: do not output the trailing newline

md5sum on linux, md5 on macOS

sha

shasum -a 256 /path/to/file
printf 'hello' | shasum -a 256

Examples

Wrap in double quotes and join with comma

echo 'hello
world' | sed 's/^/"/' | sed 's/$/"/' | tr '\n' ','

SSH, or secure shell, is an encrypted protocol used to manage and communicate with servers. You can connect to your server via SSH. There are a few different ways to login an SSH server. Public key authentication is one of the SSH authentication methods. It allows you to access a server via SSH without a password.

Creating SSH keys

List supported algorithms of SSH keys on your client and server:

$ ssh -Q key

Output

ssh-ed25519
ssh-ed25519-cert-v01@openssh.com
ssh-rsa
ssh-dss
ecdsa-sha2-nistp256
ecdsa-sha2-nistp384
ecdsa-sha2-nistp521
ssh-rsa-cert-v01@openssh.com
ssh-dss-cert-v01@openssh.com
ecdsa-sha2-nistp256-cert-v01@openssh.com
ecdsa-sha2-nistp384-cert-v01@openssh.com
ecdsa-sha2-nistp521-cert-v01@openssh.com

Choose an algorithm that supports both your client and server for generating an SSH key pair.

$ ssh-keygen -t {your_algorithm} -C "{your_comment}"

We recommend using the ed25519 algorithm to generate your SSH key. The Ed25519 was introduced on OpenSSH version 6.5. It’s using elliptic curve cryptography that offers better security with faster performance compared to DSA, ECDSA, or RSA. The RSA is even considered not safe if it’s generated with a key smaller than 2048-bit length.

$ ssh-keygen -t ed25519 -C "mac_taogenjia@gmail.com"

Output

Enter file in which to save the key (/Users/taogen/.ssh/id_ed25519):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in {filename}
Your public key has been saved in {filename}.pub

You can specify your SSH key’s filename. If you don’t want to change the filename, by default the private key filename is id_{algorithm} and the public key filename is id_{algorithm}.pub .

For security reasons, it’s best to set a passphrase for your SSH keys.

Coping the SSH public key to your server

Copying Your Public Key Using ssh-copy-id

The simplest way to copy your public key to an existing server is to use a utility called ssh-copy-id.

ssh-copy-id -i public_key_filepath username@remote_host
# or use a specific SSH port
ssh-copy-id -i public_key_filepath -p ssh_port username@remote_host

For example

ssh-copy-id -i ~/.ssh/id_ed25519_remote_1.pub -p 38157 root@xxx.xx.xxx.xxx

Copying Your public key using SSH

cat ~/.ssh/id_rsa.pub | ssh username@remote_host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
  • cat ~/.ssh/id_rsa.pub: Output the file.
  • mkdir -p ~/.ssh: Creating the ~/.ssh directory if it doesn’t exist.
  • cat >> ~/.ssh/authorized_keys: append the standard output of the previous command of the pipeline to the file ~/.ssh/authorized_keys on the remote host.

Configuring SSH

If there are multiple SSH keys on your local system, you need to configure which destination server uses which SSH key. For example, there is an SSH key for GitHub and another SSH key for a remote server.

Creating the SSH configuration file ~/.ssh/config if it doesn’t exist.

vim ~/.ssh/config

Add the config like the folowing content

# GitHub
Host github.com
User git
Port 22
Hostname github.com
IdentityFile "~/.ssh/{your_private_key}"
TCPKeepAlive yes
IdentitiesOnly yes

# Remote server
Host {remote_server_ip_address}
User {username_for_ssh}
Port {remote_server_ssh_port}
IdentityFile "~/.ssh/{your_private_key}"
TCPKeepAlive yes
IdentitiesOnly yes

SSH login with the SSH private key

If you have copied your SSH public key to the server, SSH login will automatically use your private key. Otherwise, you will need to enter the password of the remote server’s user to login.

$ ssh username@remote_host
# or use a specific port
$ ssh -p ssh_port username@remote_host

Disabling password authentication on your server

Using password-based authentication exposes your server to brute-force attacks. You can disable password authentication by updating the configuration file /etc/ssh/sshd_config.

Before disabling password authentication, make sure that you either have SSH key-based authentication configured for the root account on this server, or preferably, that you have SSH key-based authentication configured for an account on this server with sudo access.

sudo vim /etc/ssh/sshd_config

Uncomment the following line by removing # at the beginning of the line:

PasswordAuthentication no

Save and close the file when you are finished. To actually implement the changes we just made, you must restart the service.

sudo systemctl restart ssh

References

[1] How To Configure SSH Key-Based Authentication on a Linux Server

[2] Upgrade Your SSH Key to Ed25519

0%