Taogen's Blog

Stay hungry stay foolish.

Background

When I use the FreeMarker to generate text files, occurs an error: Failed to “?eval” string.

<#assign dataFromConfigJson = columnMapping.formComponentDataFromConfig?eval>

Error Info

FreeMarker template error:
Failed to "?eval" string with this error:

---begin-message---
Syntax error in ?eval-ed string in line 1, column 175:
Encountered "}", but was expecting one of:
<STRING_LITERAL>
<RAW_STRING>
"false"
"true"
<INTEGER>
<DECIMAL>
"."
".."
<DOT_DOT_LESS>
"..*"
"?"
"??"
"+"
"-"
"!"
","
"["
"]"
"("
"{"
<ID>
<TERMINATING_EXCLAM>
"+"
"-"
---end-message---

The failing expression:
==> columnMapping.formComponentDataFromConfig?eval [in template "templates/demo\\html\\test.ftl" at line 208, column 41]

----
FTL stack trace ("~" means nesting-related):
- Failed at: #assign dataFromConfigJson = columnMa... [in template "templates/demo\\html\\test.ftl" in macro "generateFormItems" at line 208, column 11]
- Reached through: @generateFormItems type="search" [in template "templates/demo\\html\\test.ftl" at line 582, column 7]
----

Solutions

Corrects the JSON string of the field of data that passed to the FreeMarker Template object.

Reasons

The JSON string is invalid.

问题1:扩展显示,外接显示器模糊

解决方案:笔记本和外接显示器的缩放比例保持一致。

Display settings -> Scale and layout

问题2:扩展显示,合上笔记本时,外接显示器变模糊

解决方案:将外接屏幕设为主显示器,并设置“仅显示外接显示器”。

Display settings -> Rearrange your display -> Click 2 for setting extended monitor

Display settings -> Multiple displays -> select “Show only on 2”, and checked “Make this main display”

问题3:笔记本使用HDMI接口连接了显示器后没声音

Solution: 声音设置中选择笔记本的扬声器作为声音输出设备。

Reason: HDMI接口是包含音频+视频两种信号的接口。当电脑的HDMI接口被使用时,系统就会默认从HDMI设备输出声音信号。

在我看来,编程它既是脑力活,也是体力活。脑力活在于如何解决未知的问题,体力活则在于一行一行地敲代码完成已知的问题。

编程中最累人的便是面对未知问题。如:未知的技术栈,未知的系统设计,未知的设计模式和类的继承结构,未知的算法实现,未知的异常等等。遇到未知问题时,我们会注意力保持高度集中,不断的失败,不断的尝试,我们绞尽脑汁想要解决这个问题,同时我们感到沮丧和急躁。然而,长时间的保持紧绷的神经和僵硬的坐姿会严重影响我们的健康状态。作为一名程序员,为了保持良好的身心健康,我们要学会如何与未知相处。

  1. 从源头出发,减少未知

我们不可能去知道所有的技术和知识。但是我们可以把自己经历过的问题,总结一下,下次遇到相似的问题,可以轻松地应对。我们也可以尽量地学习和了解一些你所处领域相关的你没有掌握的技术。

  1. 承认自己遇到了难题,并给与自己信心

每个人掌握着有限的知识,每个人都会遇到难题。遇到难题不能完全代表一个人的能力水平,难题给了我们一次成长的机会。我们尽力去解决问题,最终无论我们能不能解决这个难题,我们都会有所收获。

  1. 给与解决问题充足的时间,让问题慢慢变得明朗

解决未知问题的过程就是把未知慢慢变成已知的过程。我们在解决未知问题的过程中,经过了一段时间的思考之后,我们会发现自己有了点思路或者有了点感觉,慢慢地问题就逐渐变得越来越明朗了。

保持心平气和地去解决未知问题,这是急不来的,我们需要给大脑一些时间去理解和记忆。

Content

Upload

Upload files with Spring framework

@PostMapping(value = "/uploadFile", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public String uploadFile(@RequestParam("file") MultipartFile file, String moduleUri) throws IOException {
InputStream inputStream = file.getInputStream();
String fileName = file.getOriginalFilename();
// to save file
...
}

Upload file with Java Servlet

public class FileUploadByCommonsFileUploadServlet extends HttpServlet {
public void doPost(HttpServletRequest request, HttpServletResponse response) throws Exception {
DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
diskFileItemFactory.setSizeThreshold(MAX_MEMORY_SIZE);
diskFileItemFactory.setRepository(new File(System.getProperty("java.io.tmpdir")));
ServletFileUpload servletFileUpload = new ServletFileUpload(diskFileItemFactory);
servletFileUpload.setFileSizeMax(MAX_FILE_SIZE);
servletFileUpload.setSizeMax(MAX_FILE_SIZE);
List<FileItem> formItems = servletFileUpload.parseRequest(request);
for (FileItem item : formItems) {
if (!item.isFormField()) {
String fileName = new File(item.getName()).getName();
String filePath = new StringBuilder()
.append(UPLOAD_DIR)
.append(File.seperator)
.append(fileName)
.toString()
item.write(new File(filePath));
}
}
}
...
}

Download

Download file with spring framework

Write input streams to the OutPutStream of the HttpServletResponse

@GetMapping(value = "/download" )
public void download(HttpServletResponse response, String fileUri) {
response.setCharacterEncoding("utf-8");
response.setContentType("application/octet-stream");
response.setHeader("Content-Disposition", "attachment;fileName=\"" + URLEncoder.encode(filename, "UTF-8").replace("+", "%20") + "\"");
ServletOutputStream outputStream = response.getOutputStream();
int bufSize = 1024;
byte[] buffer = new byte[bufSize];
int len;
// get filePath by fileUri
...
// get input stream by filePath
...
while ((len = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, len);
}
inputStream.close();
}

Write file Resource to ReponseBody

@GetMapping(value = "/download" )
public ResponseEntity<Resource> download(String fileUri) {
// get filePath by fileUri
//...
String uploadDir = "D:\\upload";
Path path = Paths.get(uploadDir).toAbsolutePath().normalize();
path = path.resolve(fileUri).normalize();
Resource resource = new UrlResource(path.toUri());
String resultFileName = URLEncoder.encode(resource.getFilename().replace("%20", " "), "UTF-8").replace("+", "%20");
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType("application/octet-stream"))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + resultFileName + "\"")
.body(resource);
}

Setting Response Headers

  • content-type: application/octet-stream
  • content-disposition: attachment; filename=”test.txt”

To encode filename when download file

original file name: “中 文.txt”

filename = resource.getFilename() // 中%20文.txt

browser download filename: -%20‡.txt

1

filename = URLEncoder.encode(resource.getFilename().replace("%20", " "), "UTF-8") // %E4%B8%AD+%E6%96%87.txt

browser download filename: 中+文.txt

2

filename = URLEncoder.encode(resource.getFilename().replace("%20", " "), "UTF-8").replace("+", "%20") // %E4%B8%AD%20%E6%96%87.txt

browser download filename: 中 文.txt

When browser resolve file name, they don’t decode “+” to “ “, so we need replace “+” to “%20”.

The Example of Upload and download files in spring framework

@RestController
@RequestMapping("files")
public class MyFileController {
private static final Logger logger = LogManager.getLogger();

public static final String UPLOAD_DIR = "D:\\My Desktop\\upload";

public static final String DOWNLOAD_PREFIX = "/files/download";

@PostMapping(value = "/uploadFile", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public String uploadFile(@RequestParam("file") MultipartFile file, String moduleUri) throws IOException {
String uploadFileUri = getUploadFileUri(file, moduleUri);
String uploadFilePath = new StringBuilder()
.append(UPLOAD_DIR)
.append(File.separator)
.append(uploadFileUri)
.toString();
ensureDirectoryExist(uploadFilePath);
try (
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(
new FileOutputStream(uploadFilePath));
) {
byte[] buf = new byte[1024];
int len;
InputStream inputStream = file.getInputStream();
while ((len = inputStream.read(buf)) != -1) {
bufferedOutputStream.write(buf, 0, len);
}
}
return new StringBuilder()
.append(DOWNLOAD_PREFIX)
.append("/")
.append(uploadFileUri)
.toString();
}

private String getUploadFileUri(MultipartFile file, String moduleUri) {
String uuid = UUID.randomUUID().toString().replaceAll("-", "");
Calendar calendar = Calendar.getInstance();
StringBuilder dateUri = new StringBuilder()
.append(calendar.get(Calendar.YEAR))
.append("/")
.append(calendar.get(Calendar.MONTH) + 1)
.append("/")
.append(calendar.get(Calendar.DAY_OF_MONTH));
StringBuilder resultFileUri = new StringBuilder()
.append(moduleUri)
.append("/")
.append(dateUri)
.append("/")
.append(uuid)
.append("/")
.append(file.getOriginalFilename());
return resultFileUri.toString();
}

private void ensureDirectoryExist(String filePath) {
filePath = filePath.replace("/", File.separator).replace("\\", File.separator);
String fileDir = filePath.substring(0, filePath.lastIndexOf(File.separator));
logger.debug("fileDir: {}", fileDir);
File dir = new File(fileDir);
if (!dir.exists()) {
dir.mkdirs();
}
}

@GetMapping(value = "download/**")
public void download(HttpServletRequest request, HttpServletResponse response) throws IOException {
String servletPath = request.getServletPath();
String fileUri = servletPath.substring(DOWNLOAD_PREFIX.length());
String filePath = new StringBuilder()
.append(UPLOAD_DIR)
.append(File.separator)
.append(fileUri)
.toString();
try (
BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(filePath))
) {
ServletOutputStream outputStream = response.getOutputStream();
byte[] buf = new byte[1024];
int len;
while ((len = bufferedInputStream.read(buf)) != -1) {
outputStream.write(buf, 0, len);
}
}
}
}

Read

Read data from disk

java.io.FileInputStream

InputStream is = new FileInputStream("D:\\My Desktop\\test.txt");

org.springframework.core.io.UrlResource

Spring framework Resource Interface

Java’s standard java.net.URL class and standard handlers for various URL prefixes unfortunately are not quite adequate enough for all access to low-level resources. For example, there is no standardized URL implementation that may be used to access a resource that needs to be obtained from the classpath, or relative to a ServletContext. While it is possible to register new handlers for specialized URL prefixes (similar to existing handlers for prefixes such as http:), this is generally quite complicated, and the URL interface still lacks some desirable functionality, such as a method to check for the existence of the resource being pointed to.

String filepath = "D:\\My Desktop\\test.txt";
Path path = Paths.get(filepath).normalize();
//path.toUri().toString(): file:///D:/My%20Desktop/test.txt
Resource resource = new UrlResource(path.toUri());
InputStream is = resource.getInputStream();

java.net.URL

String filepath = "D:\\My Desktop\\test.txt";
Path path = Paths.get(filepath).normalize();
//path.toUri().toString(): file:///D:/My%20Desktop/test.txt
InputStream is = new URL(path.toUri().toString()).openStream();

Read data from Java classpath

org.springframework.core.io.ClassPathResource

// src/main/resources/application.yml
Resource resource = new ClassPathResource("application.yml");
InputStream is = resource.getInputStream();

org.springframework.core.io.ResourceLoader

@Autowired
ResourceLoader resourceLoader;

resourceLoader.getResource("classpath:application.yml");
InputStream is = resource.getInputStream();

Read data from memory

byte[] data = new String("data...").getBytes(StandardCharsets.UTF_8);
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);

Read data From HTTP URL

java.net.URL

InputStream is = new URL("http://demo.com/test.txt").openStream();

org.springframework.core.io.UrlResource

Resource resource = new UrlResource("http://demo.com/test.txt");
InputStream is = resource.getInputStream();

java.net.HttpURLConnection

URLConnection urlConnection = new URL("http://demo.com/test.txt").openConnection();
urlConnection.connect();
InputStream is = urlConnection.getInputStream();

Read data from OSS

Reference my another post: Common OSS Java SDK Usage

Write

Write data to disk

byte[] data = new String("data").getBytes(StandardCharsets.UTF_8);
String storeFilePath = "D:/My Workspace/test/test.txt";
try (
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
FileOutputStream fileOutputStream = new FileOutputStream(storeFilePath)
) {
byte[] buf = new byte[1024];
int len = 0;
while ((len = byteArrayInputStream.read(buf)) != -1) {
fileOutputStream.write(buf, 0, len);
}
} catch (IOException e) {
e.printStackTrace();
}

Write data to memory

byte[] data = new String("data").getBytes(StandardCharsets.UTF_8);
byte[] storeDataArray;
try (
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
) {
byte[] buf = new byte[1024];
int len = 0;
while ((len = byteArrayInputStream.read(buf)) != -1) {
byteArrayOutputStream.write(buf, 0, len);
}
storeDataArray = byteArrayOutputStream.toByteArray();
System.out.println(new String(storeDataArray, StandardCharsets.UTF_8));
} catch (IOException e) {
e.printStackTrace();
}

Write data to OSS

Reference my another post: Common OSS Java SDK Usage

stream

InputStream is = new FileInputStream("D:\\My Desktop\\test.txt");
new BufferedReader(new InputStreamReader(is)).lines().forEach(System.out::println);

java.nio.file.Files

Path path = Paths.get("D:\\My Desktop\\test.txt");
System.out.println(new String(Files.readAllBytes(path)));

Package files to zip

basic zip

try (
FileOutputStream fileOutputStream = new FileOutputStream("D:/My Workspace/test/test.zip");
ZipOutputStream zipOutputStream = new ZipOutputStream(fileOutputStream)
) {
int fileNumber = 3;
for (int i = 1; i <= fileNumber; i++) {
String childFilename = "file" + i + ".txt";
zipOutputStream.putNextEntry(new ZipEntry(childFilename));
zipOutputStream.write(("data" + i).getBytes(StandardCharsets.UTF_8));
zipOutputStream.closeEntry();
}
zipOutputStream.finish();
} catch (IOException e) {
e.printStackTrace();
}

nested zip

try (
FileOutputStream fileOutputStream = new FileOutputStream("D:/My Workspace/test/test.zip");
ZipOutputStream zipOutputStream = new ZipOutputStream(fileOutputStream)
) {
int fileNumber = 3;
for (int i = 1; i <= fileNumber; i++) {
String childFilename = "file" + i + ".zip";
zipOutputStream.putNextEntry(new ZipEntry(childFilename));
try (
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
ZipOutputStream nestedZipOutputStream = new ZipOutputStream(byteArrayOutputStream);
) {
int nestedNumber = 3;
for (int j = 1; j <= nestedNumber; j++) {
String nestedFileName = new StringBuilder()
.append("file").append(i)
.append("nestFile").append(j)
.append(".txt").toString();
nestedZipOutputStream.putNextEntry(new ZipEntry(nestedFileName));
nestedZipOutputStream.write(("nest file" + j).getBytes(StandardCharsets.UTF_8));
nestedZipOutputStream.closeEntry();
}
nestedZipOutputStream.finish();
zipOutputStream.write(byteArrayOutputStream.toByteArray());
}
zipOutputStream.closeEntry();
}
zipOutputStream.finish();
} catch (IOException e) {
e.printStackTrace();
}

Speed up read or write

Using buffer to decrease calls to the underlying runtime system

  1. Direct Buffering
String inputFilePath = "C:\\Users\\Taogen\\Desktop\\input.txt";
String outputFilePath = "C:\\Users\\Taogen\\Desktop\\output.txt";
try (
FileInputStream fileInputStream = new FileInputStream(inputFilePath);
FileOutputStream fileOutputStream = new FileOutputStream(outputFilePath)
) {
byte[] buf = new byte[2048];
int len = 0;
while ((len = fileInputStream.read(buf)) != -1) {
fileOutputStream.write(buf, 0, len);
}
} catch (IOException e) {
e.printStackTrace();
}
  1. Using buffered Input/output Stream
String inputFilePath = "C:\\Users\\Taogen\\Desktop\\input.txt";
String outputFilePath = "C:\\Users\\Taogen\\Desktop\\output.txt";
try (
FileInputStream fileInputStream = new FileInputStream(inputFilePath);
BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream);
FileOutputStream fileOutputStream = new FileOutputStream(outputFilePath);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
) {

int b;
while ((b=bufferedInputStream.read()) != -1) {
bufferedOutputStream.write(b);
}
} catch (IOException e) {
e.printStackTrace();
}

BufferedInputStream‘s default buffer size is int DEFAULT_BUFFER_SIZE = 8192;

  1. BufferedInputStream + Buffer array
String inputFilePath = "C:\\Users\\Taogen\\Desktop\\recovery_data.sql";
String outputFilePath = "C:\\Users\\Taogen\\Desktop\\recovery_data2.sql";
try (
FileInputStream fileInputStream = new FileInputStream(inputFilePath);
BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream);
FileOutputStream fileOutputStream = new FileOutputStream(outputFilePath);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
) {
byte[] buf = new byte[1024];
int len = 0;
while ((len = bufferedInputStream.read(buf)) != -1) {
bufferedOutputStream.write(buf, 0, len);
}
} catch (IOException e) {
e.printStackTrace();
}

Time cost: BufferedInputStream + Direct Buffer array < Direct Buffer array < Using a BufferedInputStream < Read Method.

Using BufferedInputStream rather than direct buffer is probably “right” for most applications. Because your used buffer size of direct buffer may be worse than BufferedInputStream default buffer size in speed.

Will making the buffer bigger make I/O go faster? Java buffers typically are by default 1024 or 2048 bytes long. A buffer larger than this may help speed I/O, but often by only a few percent, say 5 to 10%.

Problems

Max upload file size

Character encoding of file name

URL encode of file name

Appendixes

Temporary Files and Directories

java.io.tmpdir

System.getProperty("java.io.tmpdir")

Windows 10: C:\Users\{user}\AppData\Local\Temp\

Debian: /tmp

Create temporary file

// If you don't specify the file suffix, the default file suffix is ".tmp".
File file = File.createTempFile("temp", null);
System.out.println(file.getAbsolutePath());
file.deleteOnExit();

Common Content-Type (MIME types) Table

Java get file’s mimeType

// 1
String mimeType = Files.probeContentType(file.toPath());
// 2
String mimeType = URLConnection.guessContentTypeFromName(fileName);
// 3
FileNameMap fileNameMap = URLConnection.getFileNameMap();
String mimeType = fileNameMap.getContentTypeFor(file.getName());
// 4
MimetypesFileTypeMap fileTypeMap = new MimetypesFileTypeMap();
String mimeType = fileTypeMap.getContentType(file.getName());
Content-Type(Mime-Type) Kind of document Extension
application/octet-stream Any kind of binary data .bin
text/plain Text, (generally ASCII or ISO 8859-n) .txt
<Images>
image/bmp Windows OS/2 Bitmap Graphics .bmp
image/jpeg JPEG images .jpeg .jpg
image/png Portable Network Graphics .png
image/gif Graphics Interchange Format (GIF) .gif
image/vnd.microsoft.icon Icon format .ico
image/svg+xml Scalable Vector Graphics (SVG) .svg
<media>
audio/mpeg MP3 audio .mp3
video/mp4 MP4 video .mp4
<code>
text/css Cascading Style Sheets (CSS) .css
text/csv Comma-separated values (CSV) .csv
application/json JSON format .json
text/javascript JavaScript, JavaScript module .js .mjs
text/html HyperText Markup Language (HTML) .htm .html
<doc>
application/msword Microsoft Word .doc
application/vnd.openxmlformats-officedocument.wordprocessingml.document Microsoft Word (OpenXML) .docx
application/pdf Adobe Portable Document Format (PDF) .pdf
<archive>
application/gzip GZip Compressed Archive .gz
application/vnd.rar RAR archive .rar
application/x-tar Tape Archive (TAR) .tar
application/zip ZIP archive .zip
application/x-7z-compressed 7-zip archive .7z

Two primary MIME types are important for the role of default types:

  • text/plain is the default value for textual files. A textual file should be human-readable and must not contain binary data.
  • application/octet-stream is the default value for all other cases. An unknown file type should use this type. Browsers pay a particular care when manipulating these files, attempting to safeguard the user to prevent dangerous behaviors.

References

[1] Tuning Java I/O Performance

[2] Common MIME types - MDN Web Docs

MyBatis

MyBatis is a first class persistence framework with support for custom SQL, stored procedures and advanced mappings. MyBatis eliminates almost all of the JDBC code and manual setting of parameters and retrieval of results. MyBatis can use simple XML or Annotations for configuration and map primitives, Map interfaces and Java POJOs (Plain Old Java Objects) to database records.

  1. Add dependencies
  • mybatis
  • mysql-connector-java
<project ...>
<dependencies>
<!-- MyBatis -->
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
<version>${latest_version}</version>
</dependency>
<!-- MySQL Connector -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${latest_version}</version>
</dependency>
</dependencies>
</project>
  1. Write the mybatis-config.xml file
  • Configuring the data source.
  • Inject mappers
<configuration>
<environments default="development">
<environment id="development">
<dataSource type="POOLED">
<property name="driver" value="${MYSQL_DRIVER_CLASS}"/>
<property name="url" value="${MYSQL_URL}"/>
<property name="username" value="${MYSQL_USER}"/>
<property name="password" value="${MYSQL_PASSWD}"/>
</dataSource>
</environment>
</environments>
<mappers>
<mapper resource="mybatis/mapper/UserMapper.xml" />
</mappers>
</configuration>
  1. Write maper XML file
<mapper>
...
</mapper>
  1. Usage
  • Load mybatis-config.xml
  • Build SqlSessionFactory object
  • Get SqlSession object
  • Get mapper object
  • Call methods of mapper object
public static void main(String[] args) throws IOException {
String resource = "mybatis/mybatis-config.xml";
InputStream inputStream = Resources.getResourceAsStream(resource);
SqlSessionFactory sqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream);
try (SqlSession session = sqlSessionFactory.openSession()) {
UserMapper userMapper = session.getMapper(UserMapper.class);
User user = userMapper.selectByPrimaryKey(1);
logger.debug("user is {}", user);
}
}

MyBatis with Spring

MyBatis-Spring integrates MyBatis seamlessly with Spring. This library allows MyBatis to participate in Spring transactions, takes care of building MyBatis mappers and SqlSessions and inject them into other beans, translates MyBatis exceptions into Spring DataAccessExceptions, and finally, it lets you build your application code free of dependencies on MyBatis, Spring or MyBatis-Spring.

  1. Add dependencies
  • spring related
  • mybatis
  • mybatis-spring
  • mysql-connector-java
<!-- Spring -->
...
<!-- mybatis -->
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
<version>${mybatis.version}</version>
</dependency>
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis-spring</artifactId>
<version>${mybatis.spring.version}</version>
</dependency>
<!-- mysql -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql.jdbc.version}</version>
</dependency>
  1. Write spring-mybatis.xml configuration file
  • Configuring dataSource
  • Configuring sqlSessionFactory for specifying mapper xml files location.
  • Configuring MapperScannerConfigurer for specifying mapper interface Java files location.
<bean id="dataSource" class="com.alibaba.druid.pool.DruidDataSource"
init-method="init" destroy-method="close">
...
</bean>
<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="mapperLocations" value="classpath:mybatis/mapper/*.xml"/>
</bean>
<bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
<property name="basePackage" value="com.taogen.example.dao"/>
<property name="sqlSessionFactoryBeanName" value="sqlSessionFactory"/>
</bean>
  1. Write maper XML file
<mapper>
...
</mapper>
  1. Usage
  • Call Dao interface method
@Service
public class UserServiceImpl implements UserService {
@Autowired
private UserDao userDao;

@Override
public List<User> listAllUsers() {
return userDao.listAllUsers();
}
...
}

MyBatis with Spring Boot

The MyBatis-Spring-Boot-Starter help you build quickly MyBatis applications on top of the Spring Boot.

By using this module you will achieve:

  • Build standalone applications
  • Reduce the boilerplate to almost zero
  • Less XML configuration

MyBatis-Spring-Boot-Starter will:

  • Autodetect an existing DataSource
  • Will create and register an instance of a SqlSessionFactory passing that DataSource as an input using the SqlSessionFactoryBean
  • Will create and register an instance of a SqlSessionTemplate got out of the SqlSessionFactory
  • Auto-scan your mappers, link them to the SqlSessionTemplate and register them to Spring context so they can be injected into your beans
  1. Add dependencies
  • mybatis-spring-boot-starter
  • mysql-connector-java
<!-- spring boot -->
...
<!-- data access -->
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
  1. Write Spring Boot configuration file application.yml
  • Configuring data sources
  • Configuring mybatis for specifying mapper xml files location.
spring:
datasource:
url: jdbc:mysql://localhost:3306/my_test?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8
username: root
password: root
driverClassName: com.mysql.cj.jdbc.Driver
mybatis:
mapper-locations: classpath:mybatis/mapper/*.xml
type-aliases-package: com.taogen.demo.springbootcrud.module.*.entity
  1. Write maper XML file
<mapper>
...
</mapper>
  1. Definition mapper interfaces

The MyBatis-Spring-Boot-Starter will search, by default, for mappers marked with the @Mapper annotation.

@Mapper
public interface UserDao {
}

You may want to specify a custom annotation or a marker interface for scanning. If so, you must use the @MapperScan annotation.

@MapperScan(value = "com.taogen.example.dao")

The value attribute is a alias for the basePackages attribute.

  1. Usage
  • Call Dao interface method
@Service
public class UserServiceImpl implements UserService {
@Autowired
private UserDao userDao;

@Override
public List<User> listAllUsers() {
return userDao.listAllUsers();
}
...
}

MyBatis Plus with Spring Boot

MyBatis-Plus (opens new window)(简称 MP)是一个 MyBatis (opens new window)的增强工具,在 MyBatis 的基础上只做增强不做改变,为简化开发、提高效率而生。

  • 无侵入:只做增强不做改变,引入它不会对现有工程产生影响,如丝般顺滑。
  • 损耗小:启动即会自动注入基本 CURD,性能基本无损耗,直接面向对象操作。
  • 强大的 CRUD 操作:内置通用 Mapper、通用 Service,仅仅通过少量配置即可实现单表大部分 CRUD 操作,更有强大的条件构造器,满足各类使用需求。
  • 支持 Lambda 形式调用:通过 Lambda 表达式,方便的编写各类查询条件,无需再担心字段写错。
  1. Add dependencies
  • mybatis-plus-boot-starter
  • mysql-connector-java
<!-- spring boot -->
...
<!-- data access -->
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-boot-starter</artifactId>
<version>3.4.3</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
  1. Write Spring Boot configuration file application.yml
  • Configuring data sources
  • Configuring mybatis for specifying mapper xml files location. (If you just use mybatis plus service and dao method, you needn’t configure this.)
spring:
datasource:
url: jdbc:mysql://localhost:3306/my_test?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8
username: root
password: root
driverClassName: com.mysql.cj.jdbc.Driver
mybatis:
mapper-locations: classpath:mybatis/mapper/*.xml
type-aliases-package: com.taogen.demo.springbootcrud.module.*.entity
  1. Write maper XML file

(If you just use mybatis plus service and dao method, you needn’t this.)

<mapper>
...
</mapper>
  1. Definition mapper interfaces

The MyBatis-Spring-Boot-Starter will search, by default, for mappers marked with the @Mapper annotation.

import com.baomidou.mybatisplus.core.mapper.BaseMapper;

@Mapper
public interface UserDao extends BaseMapper<User> {
}

You may want to specify a custom annotation or a marker interface for scanning. If so, you must use the @MapperScan annotation.

@MapperScan(value = "com.taogen.example.dao")

The value attribute is a alias for the basePackages attribute.

  1. extends MyBatis Plus class

dao

import com.baomidou.mybatisplus.core.mapper.BaseMapper;

@Mapper
public interface UserDao extends BaseMapper<User> {
}

service

import com.baomidou.mybatisplus.extension.service.IService;

public interface UserService extends IService<User> {
}
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;

@Service
public class UserServiceImpl extends ServiceImpl<UserDao, User> implements UserService {
}
  1. Usage
  • Call MyBatis Plus CRUD methods
QueryWrapper<User> queryWrapper = new QueryWrapper();
List<User> userList = userService.list(queryWrapper);

More MyBatis Plus method reference CRUD接口

References

[1] mybatis

[2] mybatis-spring

[3] mybatis-spring-boot-autoconfigure

[4] MyBatis-Plus 快速开始

Today I try to delete a directory with 485,000+ files on Linux server. It shows the error message below.

/bin/rm: Argument list too long.

If you’re trying to delete a very large number of files in a single directory at one time, you will probably run into this error. The problem is that when you type something like rm -f /data/*. So how do we working with directories with a large number of files? Let talk about it below.

Show first files

To show first n files in the directory instead of to show all files.

ls -U | head -10

Count the number of files

ls -f | wc -l

Remove files

Using for Loop

for f in *.pdf; do echo rm "$f"; done
for f in *.pdf; do rm "$f"; done

Using Find

Using find -delete. (2000 files/second, it is about three time faster rm)

find . -maxdepth 1 -type f -name "$.split-*" -print -delete

Using find and xargs. To find every file and pass them one-by-one to the “rm” command (not recommend, xargs is dangerous*)

find . -type f -exec rm -v {} \;
find . -maxdepth 1 -name "$.split-*" -print0 | xargs -0 rm

View and Edit large files

Open the 4.14 GB sql file

Computer

  • ThinkPad T14s
  • Power mode: Best performance
  • Processor: 10th Generation Intel® Core™ i5-10210U Processor (1.60 GHz, up to 4.20 GHz with Turbo Boost, 4 Cores, 8 Threads, 6 MB Cache)
  • Memory: 16GB
  • Hard Drive: 512 GB PCIe SSD

Result

  • vim: opened about 13 seconds. saved about 20 seconds.
  • sublime: opened 3 min 30 seconds. saved about 13 seconds.
  • less: opened instantly. can’t edit.
  • Intellij IDEA: opened about 3 seconds. can’t edit. and tip “The file is too large. Read-only mode.”
  • emacs: opened about 15 seconds. saving crashed.
  • Nodepad++: Can’t open. Error: file is too big opend by Notepad++.
  • VS code: Can’t open. open crashed

Best overall: vim.

Best for view: less or Intellij IDEA.

Export data from or Import data to remote MySQL server

Export 4.14 GB data from remote MySQL server

Cost time: 3 hours.

best exporting data at night.

软件开发并不只是写代码。一次提交的代码可能很少,但可能也需要做大量其它相关的工作。写代码的时间只是软件开发工作的一部分,还要很多其它事情要做,如:分析和理解需求,设计系统和功能,尝试多种解决方案后找到合适的解决方案并实现代码,与其他同事进行对接和沟通,测试代码和修复bug等。

我认为要控制好软件开发的进度,需要做好以下这两件事:

  1. 时间规划和任务划分。正确地估算,设置合理的预期时间,调整开发时间安排。这个需要比较丰富的开发经验才能很好地估算。
  2. 更高效、稳定地完成任务。这个需要平时的积累。

接下来,将针对第2点“如何更高效、稳定地完成任务”进行详细地描述。

如何更高效、稳定地开发

需求分析

  1. 明确需求。有需求规格说明书文档。有文档方便快速回顾。
  2. 明确 UI 或原型。
  3. 对需求的理解要清晰、明确,有问题及时沟通。需求是软件开发的目标,目标不清楚,只会浪费时间做无用功。这是非常重要关键的,不要不好意思去问或者拖延不去问。

系统设计和详细设计

  1. 编写系统设计文档,详细设计文档。有文档方便快速回顾。
  2. 平时可以积累一些设计的模板。系统设计无非是架构设计、数据库设计、API 设计等。按照规范去做,刚开始可能会很慢,多做几次就好一些。
  3. 做好了软件设计,心里会比较有底,能够大致知道有哪些事情要做和自己所处的进度。知道了自己的进度,就能很好地进行调整,而不是前面很松懈,后面一直加班追赶进度。

代码实现

  1. 平时积累学习一些相关的技术。防止开发过程中,边学习边开发,影响进度。
  2. 平时积累一些代码。通用工具代码、功能实现代码。可以更快、更省力地写代码。
  3. 平时积累一些代码问题解决的笔记或博客。遇到相似问题可以有参考,防止卡住进度。
  4. 编写整洁、易读、方便扩展的代码。方便快速地修改、扩展代码,以及减少潜在问题。
  5. 快速实现,不断迭代。初期不要过分追求完美的代码。“过早优化是万恶之源”。

测试

  1. 写好测试用例文档,方便进行完整、严格的测试。平时可以积累一些常见功能的测试用例。
  2. 提交代码前,通过测试用例,减少修复 bug 的时间。
  3. 修改代码,做一些回归测试,减少修复 bug 的时间。
  4. 写单元测试或者使用测试驱动开发,可以更自信地重构和优化代码,防止重构时引入新的 bug。

Iconfont 是阿里妈妈MUX倾力打造的矢量图标管理、交流平台。

设计师将图标上传到 iconfont 平台,用户可以自定义下载多种格式的icon,平台也可将图标转换为字体,便于前端工程师自由调整与调用。

Unicode 引用

  • 支持按字体的方式去动态调整图标大小,颜色等等。
  • 默认情况下不支持多色,直接添加多色图标会自动去色。

第一步:引入项目的样式文件

找到你的项目:资源管理 –> 我的项目 –> 我参与的项目 –> xxx项目

选择使用 Unicode 外链或者将代码下载至本地

/* 引入外链 */
@font-face {
font-family: 'myIcon'; /*自定义,默认为项目名称*/
src: url('//at.alicdn.com/t/c/font_xxx.woff2?t=1686623243724') format('woff2'),
url('//at.alicdn.com/t/c/font_xxx.woff?t=1686623243724') format('woff'),
url('//at.alicdn.com/t/c/font_xxx.ttf?t=1686623243724') format('truetype');
}
/* 引入本地文件 */
@font-face {
font-family: 'myIcon'; /*自定义,默认为项目名称*/
src: url('iconfont.woff2?t=1625109690491') format('woff2'),
url('iconfont.woff?t=1625109690491') format('woff'),
url('iconfont.ttf?t=1625109690491') format('truetype');
}

第二步:定义全局的 unicode 图标样式

/* 自定义 className */
.myIconFont {
font-family: "myIcon" !important; /* 需要与你的@font-face中的font-family保持一致 */
font-size: 16px;
font-style: normal;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}

第三步:挑选相应图标并获取字体编码,应用于页面

<span class="myIconFont">&#x33;</span>
<span class="myIconFont">&#xe718;</span>
<i class="myIconFont">&#xe718;</i>

修改图标大小:设置 font-size

<span class="myIconFont" style="font-size: 500px;">&#x33;</span>

修改图标颜色:设置 color

<span class="myIconFont" style="color: #8BC34A;">&#x33;</span>

Font Class 引用

font-class 是 Unicode 使用方式的一种变种,主要是解决 Unicode 书写不直观,语意不明确的问题。

与 Unicode 使用方式相比,具有如下特点:

  • 相比于 Unicode 语意明确,书写更直观。可以很容易分辨这个 icon 是什么。
  • 因为使用 class 来定义图标,所以当要替换图标时,只需要修改 class 里面的 Unicode 引用。

第一步:引入项目的样式文件

找到你的项目:资源管理 –> 我的项目 –> 我参与的项目 –> xxx项目

选择使用 Font class 外链或者将代码下载至本地

<!-- 引入外链 -->
<link rel="stylesheet" type="text/css" href="//at.alicdn.com/t/font_xxx.css">
<!-- 引入本地文件 -->
<link rel="stylesheet" type="text/css" href="./iconfont.css">

第二步:挑选相应图标并获取类名,应用于页面

<!-- 注意:第一个 class 需要与 iconfont.css 文件中的 @font-face {font-family: "myIconFont"} 保持一致,默认项目名,可自定义。第二个 class 是icon+图标的名称 -->
<span class="myIconFont icon-xxx"></span>

改变图标的大小:重写 font-size 属性

<span class="myIconFont icon-xxx" style="font-size: 500px;"></span>

Symbol 引用

这是一种全新的使用方式,应该说这才是未来的主流,也是平台目前推荐的用法。这种用法其实是做了一个 SVG 的集合,与另外两种相比具有如下特点:

  • 支持多色图标了,不再受单色限制。
  • 通过一些技巧,支持像字体那样,通过 font-size, color 来调整样式。
  • 兼容性较差,支持 IE9+,及现代浏览器。
  • 浏览器渲染 SVG 的性能一般,还不如 png。

第一步:引入项目的样式文件

找到你的项目:资源管理 –> 我的项目 –> 我参与的项目 –> xxx项目

选择使用 Symbol 外链或者将代码下载至本地

<!-- 引入外链 -->
<script type="text/javascript" src="//at.alicdn.com/t/c/font_xxx.js">
<!-- 引入本地文件 -->
<script type="text/javascript" src="./iconfont.js"></script>

第二步:定义全局的 symbol 图标样式

<style>
/*自定义样式class名称*/
.myIcon {
width: 1em;
height: 1em;
vertical-align: -0.15em;
fill: currentColor;
overflow: hidden;
}
</style>

第三步:挑选相应图标并获取类名,应用于页面

<!-- 样式名称与上一步定义的保持一致 -->
<svg class="myIcon" aria-hidden="true">
<use xlink:href="#icon-xxx"></use>
</svg>

改变图标的大小:设置 font-size 属性

<svg class="myIcon" aria-hidden="true" style="font-size: 500px;">
<use xlink:href="#icon-xxx"></use>
</svg>

改变图标的大小:同时设置 width 和 height 属性(只设置一个没有作用)

<svg class="myIcon" aria-hidden="true" style="width: 300px; height: 300px;">
<use xlink:href="#icon-xxx"></use>
</svg>

References

Background

Using HTTP domain visit my website is ok, but HTTPS not.

Error Info

ERR_TIMED_OUT

This site can’t be reached
xxx.xxx.com took too long to respond.
Try:
- Checking the connection
- Checking the proxy and the firewall
- Running Windows Network Diagnostics
ERR_TIMED_OUT

Solutions

ERR_TIMED_OUT represents this is a connection error, not SSL certificate errors.

Common Solution

  • Check whether the server IP or HTTP domain can be visited. If ok, the connection is OK.
  • Check whether the server reversed proxy is right, and server firewall is open 80 and 443.
  • Check whether the client browser and client network is right.

Current HTTP is work represented the connection is ok. And client browser can visit other HTTPS websites, represented client is ok.

You need to check your server reversed proxy is right, and ports 80 and 443 of your server firewall are open .

Reasons

My server’s port 443 is not open.

References

[1] https timeout while http works

0%