Taogen's Blog

Stay hungry stay foolish.

Apache POI is a Java API for Microsoft Documents processing. It provides pure Java libraries for reading and writing files in Microsoft Office formats, such as Word, PowerPoint and Excel. The code examples in this post are based on Apache POI v5.0.0.

<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>5.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>5.0.0</version>
</dependency>

poi is for Excel 97-2003 workbook.

poi-ooxml is for Excel 2007+ workbook.

Word

Write Data into Word

A basic use example

public static void main(String[] args) throws IOException {
XWPFDocument document = new XWPFDocument();
XWPFParagraph paragraph = document.createParagraph();
XWPFRun run = paragraph.createRun();
run.setText("Hello World");
run.addBreak();
run.setText("Apache POI");
run.setBold(true);
run.setFontSize(20);
String filePath = "C:\\Users\\Taogen\\Desktop\\hello.docx";
document.write(new FileOutputStream(filePath));
}

Alignment

XWPFParagraph paragraph = doc.createParagraph();

paragraph.setAlignment(ParagraphAlignment.CENTER);

Indentation

XWPFParagraph paragraph = document.createParagraph();

Integer indentationFirstLine; // 1/20th point
paragraph.setIndentationFirstLine(indentationFirstLine);
Integer indentationLeft; // 1/20th point
paragraph.setIndentationLeft(indentationLeft);
Integer indentationRight; // 1/20th point
paragraph.setIndentationRight();

Spacing

XWPFParagraph paragraph = document.createParagraph();

Integer spacingBefore; // 1/20th point
paragraph.setSpacingBefore(spacingBefore);
Integer spacingBeforeLines; // 1/100th line
paragraph.setSpacingBeforeLines(spacingBeforeLines);
Integer spacingAfter; // 1/20th point
paragraph.setSpacingAfter(spacingAfter);
Integer spacingAfterLines; // 1/100th line
paragraph.setSpacingAfterLines(spacingAfterLines);
Integer spacingBetween; // 1 line or 1 point, It depends on what LineSpacingRule used
paragraph.setSpacingBetween(spacingBetween, LineSpacingRule.AUTO);
paragraph.setSpacingBetween(spacingBetween, LineSpacingRule.EXACT);

Font

XWPFParagraph paragraph = document.createParagraph();

XWPFRun run = paragraph.createRun();
Integer fontSize; // 1 point
run.setFontSize(fontSize);
String fontFamily = "Calibri";
run.setFontFamily(fontFamily);
String color = "000000"; // RGB string
run.setColor(color);
run.setBold(false);
run.setItalic(false);
UnderlinePatterns underline;
run.setUnderline(underline);
run.setStrikeThrough(false);
Integer characterSpacing; // 1/20th points
run.setCharacterSpacing(characterSpacing)

Text

XWPFParagraph paragraph = document.createParagraph();

XWPFRun run = paragraph.createRun();
String text = "Hello World!";
run.setText(text);

Add newline

run.addBreak();

Insert Images

XWPFParagraph imageParagraph = document.createParagraph();

imageParagraph.setAlignment(ParagraphAlignment.CENTER);
XWPFRun imageRun = imageParagraph.createRun();
Integer textPosition = 100; // 1/2nd points
imageRun.setTextPosition(textPosition);
byte[] imageBytes = IOUtils.toByteArray(bufferedInputStream);
BufferedImage bufferedImage = ImageIO.read(new ByteArrayInputStream(imageBytes));
int imageWidth = bufferedImage.getWidth();
int imageHeight = bufferedImage.getHeight();
double scalePercent = 0.2;
int scaledWidth = (int) (imageWidth * scalePercent);
int scaledHeight = (int) (imageHeight * scalePercent);
String fileName = imageFilePath.substring(imageFilePath.lastIndexOf(File.separator) + 1);
imageRun.addPicture(new ByteArrayInputStream(imageBytes),
getImageFormat(fileName),
fileName,
Units.toEMU(scaledWidth),
Units.toEMU(scaledHeight));
document.write(bufferedOutputStream);

Insert Tables

TODO

insert Charts

TODO

Insert HTML

Excel

Write Data into Excel

A basic use example

public static void main(String[] args) throws IOException {
XSSFWorkbook workbook = new XSSFWorkbook();
XSSFSheet sheet = workbook.createSheet();
XSSFRow row = sheet.createRow(0);
XSSFCell cell = row.createCell(0);
cell.setCellValue("Hello World");
workbook.write(new FileOutputStream("D:\\test.xlsx"));
}

Column

Column width

sheet.setColumnWidth(0, 15 * 256); // unit: 1/256 character

Row

Row height

XSSFRow row = sheet.createRow(rowNum);
row.setHeight((short) (row.getHeight() * 20)); // unit: 1/20 point

Cell

XSSFCell

XSSFCell cell = row.createCell(0);
// value
cell.setCellValue("Hello World");

XSSFCellStyle cellStyle = workbook.createCellStyle();
// align
cellStyle.setAlignment(HorizontalAlignment.CENTER);
cellStyle.setVerticalAlignment(VerticalAlignment.CENTER);
// wrap text
cellStyle.setWrapText(true);
// backgroud color
cellStyle.setFillForegroundColor(IndexedColors.YELLOW.getIndex());
cellStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND);
// custom backgroud color. note: use XSSFCellStyle object.
byte[] rgb = {(byte) 155, (byte) 194, (byte) 230};
cellStyle.setFillForegroundColor(new XSSFColor(rgb, new DefaultIndexedColorMap()));
cellStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND);
// border
cellStyle.setBorderTop(BorderStyle.MEDIUM);
cellStyle.setBorderBottom(BorderStyle.MEDIUM);
cellStyle.setBorderLeft(BorderStyle.MEDIUM);
cellStyle.setBorderRight(BorderStyle.MEDIUM);
cellStyle.setTopBorderColor(IndexedColors.BLUE.getIndex());
cellStyle.setBottomBorderColor(IndexedColors.BLUE.getIndex());

// font
Font font = workbook.createFont();
font.setFontName("仿宋");
font.setFontHeightInPoints((short) 14);
font.setColor(IndexedColors.RED.getIndex());
font.setBold(true);
font.setItalic(true);
font.setUnderline(FontUnderline.SINGLE);
font.setStrikeout(true);
cellStyle.setFont(font);

cell.setCellStyle(cellStyle);

HSSFCell

// custom backgroud color
HSSFPalette palette = workbook.getCustomPalette();
// the palette index, between 0x8 to 0x40 inclusive
short colorIndex = 8;
palette.setColorAtIndex(colorIndex, (byte) 189, (byte) 215, (byte) 238);
cellStyle.setFillForegroundColor(palette.getColor(colorIndex).getIndex());
cellStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND);
XSSFCell cell = sheet.createRow(rowNum).createCell(columnNum);
cell.setCellValue("Open " + url);
XSSFWorkbook workbook = sheet.getWorkbook();
CreationHelper createHelper = workbook.getCreationHelper();
XSSFHyperlink hyperlink = (XSSFHyperlink) createHelper.createHyperlink(HyperlinkType.URL);
hyperlink.setAddress(url);
cell.setHyperlink(hyperlink);
XSSFCellStyle cellStyle = workbook.createCellStyle();
XSSFFont font = workbook.createFont();
font.setUnderline(FontUnderline.SINGLE);
font.setColor(IndexedColors.BLUE.getIndex());
cellStyle.setFont(font);
cell.setCellStyle(cellStyle);

Rich Text String

XSSFCell cell = sheet.createRow(rowNum).createCell(columnNum);
XSSFFont font = new XSSFFont();
font.setColor(IndexedColors.RED.getIndex());
XSSFFont font2 = new XSSFFont();
font2.setColor(IndexedColors.GREEN.getIndex());
RichTextString richTextString = sheet.getWorkbook().getCreationHelper()
.createRichTextString("hello world");
richTextString.applyFont(0, 5, font);
richTextString.applyFont(6, 11, font2);
cell.setCellValue(richTextString);

Merge Cell

sheet.addMergedRegion(new CellRangeAddress(startRow, endRow, startCol, endCol));
sheet.addMergedRegion(CellRangeAddress.valueOf("A1:B3"));

Note: If set cell styles for merged cells, you should set the leftmost or topmost cell.

Cell Utility

get cell

Cell cell = CellUtil.getCell(CellUtil.getRow(rowNum, sheet), columnNum);

set cell style

// align
CellUtil.setAlignment(cell, HorizontalAlignment.LEFT);
CellUtil.setVerticalAlignment(cell, VerticalAlignment.TOP);
// wrap text
CellUtil.setCellStyleProperty(cell, CellUtil.WRAP_TEXT, true);
// font
CellUtil.setFont(cell, font);

Insert Images

InputStream inputStream = new FileInputStream("C:\\Users\\Taogen\\Desktop\\demo.png");
byte[] bytes = IOUtils.toByteArray(inputStream);
int pictureIdx = workbook.addPicture(bytes, Workbook.PICTURE_TYPE_PNG);
inputStream.close();
//Returns an object that handles instantiating concrete classes
CreationHelper helper = workbook.getCreationHelper();
//Creates the top-level drawing patriarch.
Drawing drawing = sheet.createDrawingPatriarch();
//Create an anchor that is attached to the worksheet
ClientAnchor anchor = helper.createClientAnchor();
//set top-left corner for the image
anchor.setCol1(1);
anchor.setRow1(2);
//Creates a picture
Picture picture = drawing.createPicture(anchor, pictureIdx);
//Reset the image size
double scale = 0.2;
picture.resize(scale); // or picture.resize() original size

Read Data from Excel

Workbook workbook = new XSSFWorkbook(new File(filePath));
DataFormatter formatter = new DataFormatter();
Sheet sheet = workbook.getSheetAt(0);
Iterator<Row> iterator = sheet.iterator();
Row row;
while (iterator.hasNext()) {
row = iterator.next();
for (Cell cell : row) {
System.out.print(formatter.formatCellValue(cell) + "\t");
}
System.out.println();
}

Appendixes

Javadocs

Excel (XSSF)

Word (XWPF)

In the SQL world, order is not an inherent property of a set of data. Thus, you get no guarantees from your RDBMS that your data will come back in a certain order – or even in a consistent order – unless you query your data with an ORDER BY clause.

If an <order by clause> is not specified, then the ordering of the rows is implementation-dependent.

Default Orders

MySQL Server v5.6, InnoDB

If select fields only in unique key/index, the default order is ordered by unique key/index.

if select fields only in primary key, the default order is ordered by unique key/index or primary key

if select fields contain a field that is not in the primary key, unique key, and index, the default order is ordered by the primary key.

Suggestions

Do not depend on order when ORDER BY is missing.

Always specify ORDER BY if you want a particular order.

References

[1] What is The Default Sort Order of SELECTS with no ORDER BY Clause?

[2] What is the default order of records for a SELECT statement in MySQL?

[3] SQL: What is the default Order By of queries?

Ways of Exporting Requests

Synchronized Export

If to exported data is not large and the exporting process can be finished in a few seconds, we can just use the synchronized exporting. Send an export request, and then download the organized data file.

Asynchronized Export

If to exported data is too large, exporting will cost a lot of time. So we need to use the asynchronized exporting. Send an export request, view the schedule of the export, wait for the handling of exporting files to be finished, and download the organized data file.

Ways of Export implementations

  1. Write data into Java servlet response output stream.

  2. Write data into file and store in user temporary directory user.dir, and return download file URI. Delete file when download is finished.

  3. Upload data to OSS, and return download file URL.

The Limit of Max Size of Exported Data

Sometimes we need consider to setting the max size of data for exporting.

Fetch External Data

Fetch data from relational databases

Fetch static files with HTTP URLs

Build Exported Files

Exported File Types

  • Office Excel
  • Office Doc
  • Zip

Optimization

Database

  • SQL optimization. 1) Only query required columns. 2) Add index for query.

Cache

  • Cache rarely modified database data to Redis or memory.

Network IO

Fetch Database data

  • Fetch rows with multiple threads.
  • Each time to fetch a limited number of rows. It depends on the data size of a row. E.g. for the small size of a row, you can fetch 500 rows at a time.

Fetch Static Files by URL

  • Fetch files with multiple threads. E.g. 20 threads.
  • Caching files in the temporary directory.
  • Using blocking NIO, non-blocking NIO, or NIO2.

Compression

  • Compress images.
  • Compress text files.
  • Compress binary files.

Disk IO

  • When using traditional blocking IO, reading and writing files should use buffered Input/output wrapper class (E.g. BufferedInputStream) or read from/write to direct buffering array.

    // buffered Input/output wrapper class
    FileInputStream fis = new FileInputStream(filepath);
    BufferedInputStream bis = new BufferedInputStream(fis);
    // direct buffering array
    FileInputStream fis = new FileInputStream(filepath);
    byte buf[] = new byte[2048];
    int len;
    while ((len = fis.read(buf)) != -1) {}
  • (Optional) Using blocking NIO, non-blocking NIO, or NIO2. The Java NIO package offers the possibility to transfer bytes between two Channels without buffering them into the application memory. In single thread environment, traditional IO is better. NIO is used not because it’s faster but because it has better scalability especially there are amounts of clients.

    BufferedReader reader = null;
    BufferedWriter writer = null;
    try {
    Path inputFile = Paths.get(inputPath);
    Path outputFile = Paths.get(outputPath);
    reader = Files.newBufferedReader(inputFile, Charset.defaultCharset());
    writer = Files.newBufferedWriter(outputFile, Charset.defaultCharset());
    String lineFromFile = "";
    while ((lineFromFile = reader.readLine()) != null) {
    writer.append(lineFromFile);
    }
    } catch (IOException e) {
    System.out.println(e);
    } finally {
    reader.close();
    writer.close();
    }
    RandomAccessFile input = new RandomAccessFile(inputPath, "r");
    FileChannel inChannel = input.getChannel();
    RandomAccessFile output = new RandomAccessFile(outputPath, "rw");
    FileChannel outChannel = output.getChannel();
    int bufferSize = 1024;
    if (bufferSize > inChannel.size()) {
    bufferSize = (int) inChannel.size();
    }
    ByteBuffer buff = ByteBuffer.allocate(bufferSize);
    buff.clear();
    while (inChannel.read(buff) != -1) {
    buff.flip();
    outChannel.write(buff);
    buff.compact();
    }
    buff.flip();
    while (buff.hasRemaining()) {
    outChannel.write(buff);
    }
    inChannel.close();
    outChannel.close();

Build Files

  • Build exported files with multiple threads. And then pack them into a zip file.

Notice

Out of memory

Be careful when loading large of data into memory.

Memory leak

Don’t forget close resources. E.g. thread pool, input/output streams and so on.

References

[1] Tuning Java I/O Performance - Oracle

[2] how to choose java nio vs io? - Stack Overflow

[3] NIO Performance Improvement compared to traditional IO in Java - Stack Overflow

[4] FileChannel, ByteBuffer, Memory-Mapped I/O, Locks

关于学习编程技术,我认为,没有最好的方法,只有适合自己当前水平状态的、可以达成自己目标的学习方法。每个人有不同的适应自己的学习方法,每个阶段有不同的学习方法,学习的目的不同使用的方法也不同。简而言之,方法不是固定不变的。

学习方法一般有:

  • 看官方文档。
  • 看书籍。
  • 看博客、教程等技术文章。
  • 看视频教程(倍速)。
  • 看源码。
  • 跟着教程敲代码,写demo,做练习。
  • 在真实的项目中应用该技术。

学习的目的一般有:

  • 入门,学会基本概念和使用。
  • 熟悉,查漏补缺,加强熟练度。
  • 精通,深入理解架构设计和代码实现原理。

学习主要考虑两个问题:质量和速度。有的时候需要优先考虑质量,有的时候则需要优先考虑速度。

(一)入门

入门在于理解,表现为能够使用简单的几句话将这门技术的核心概念描述清楚。

如果是比较急(速度优先),则优先考虑看入门视频教程,或者入门博客教程等。文档和书籍一般都会比较权威、严谨和全面详细,然而缺点就是比较枯燥和繁琐,概念特别多,容易抓不住重点。然而入门视频教程通过通俗易懂的讲解、简单的示例代码、和抓重点内容,能够让你更好地快速入门。

如果不是很着急,可以先尝试看官方文档,如果官方文档写得对新手不太友好,可以尝试找一些初级的书籍看看。如果文档和书籍都看不下去,最后,再去找一些入门视频教程和博客文章去看。如 Spring Security 的官方文档就对新手不太友好。

另外,入门的话,跟着教程敲 demo 代码也是很有必要的。虽然 demo 代码都比较简单,自己能够看得懂,但是还是要自己敲一遍。至少能够加深一些印象。

注意事项:

  • 入门时尽量不记或者少记笔记,在大量练习使用后,再做笔记。这时候才知道哪些是重要的或经常使用的。

(二)熟悉

熟悉在于大量练习,表现为对这门技术的大部分的功能用法(如:注解、配置等)和 API 调用可以熟稔于心、信手拈来。

如果想要熟悉一个技术,首先需要认真地多看几遍权威的资料的,如官方文档和书籍。另外,大量的练习也是必不可少的。站在岸上学不会游泳,可以做一些 toy project 或者在真实的项目中应用。

在准备面试的时候,我们通常需要去背很多知识点。因为这些知识点平时用的很少,但面试需要我们流畅地表达出来。想要对某些技术知识点做到脱口而出,要么死记硬背,要么大量使用。所以面试的时候,对于用的少的知识,要去记忆和背诵。虽然短时间能记住很多,但没使用过或使用得很少,复述出来还是不太流畅。

(三)精通

何为精通?这似乎是个很模糊的概念,很难去定义。就算是领域高手也很少说自己精通某个技术。似乎精通没有极限,我们只能在通往精通的道路上不断前进。学无止境,不必为了精通而精通。以目的和兴趣去驱动,不断提高技术水平。

(四)最后

知易行难,唯有行动,才能改变。不断努力,不断成长。

References

[1] 学习编程是看文档还是看视频教程

HTML 元素原则上应该保持唯一,但没有硬性要求。

存在多个相同的 id 元素时,CSS 对所有元素都生效。

<style>
#myId {
color:red;
}
</style>
<div id="myId">
div1
</div>
<div id="myId">
div2
</div>

然而,JavaScript 和 jQuery 都只获取多个相同的 id 元素的第一个元素。

document.getElementById("myId").innerHTML //div1
$("#myId").html() //div1

Background

Using List<Integer> ids to receive a form parameter of HTTP request. The size of the parameter ids is over 255.

Error Info

org.springframework.beans.InvalidPropertyException: Invalid property 'ids[256]' of bean class [com.fantasi.manage.console.web.modules.recovery.dto.RecoveryGroupDto]: Invalid list index in property path 'ids[256]'; nested exception is java.lang.IndexOutOfBoundsException: Index: 256, Size: 256. 

Solutions

To override init Binder method and configure the limit size which needs so that you can pass those many objects to your controller classes.

@Controller
public class MyController {

@InitBinder
public void initBinder(WebDataBinder dataBinder) {
dataBinder.setAutoGrowCollectionLimit(500);
}

...
}

@InitBinder plays the role to identify the methods which initialize the WebDataBinder.

WebDataBinder is a DataBinder that binds request parameter to JavaBean objects.

Reasons

By default spring only maps only 255 objects to the java bean. Spring developers has done this to prevent OutOfMemory problem.

References

[1] Spring initBinder for Handling Large List of Java Objects

Encode

Base64

String text = "hello 世界";
String encodeText = Base64.getEncoder().encodeToString(text.getBytes(StandardCharsets.UTF_8));
System.out.println(encodeText);
String decodeText = new String(Base64.getDecoder().decode(encodeText), StandardCharsets.UTF_8);
System.out.println(decodeText);

URL Encode

String text = "hello 世界";
String encodeText = URLEncoder.encode(text, StandardCharsets.UTF_8.name());
System.out.println(encodeText);
String decodeText = URLDecoder.decode(encodeText, StandardCharsets.UTF_8.name());
System.out.println(decodeText);

Hash Algorithms

MD5

MD5 (Message Digest 5)

String text = "123456";
MessageDigest messageDigest = MessageDigest.getInstance("MD5");
messageDigest.reset();
messageDigest.update(text.getBytes(StandardCharsets.UTF_8));
byte[] digest = messageDigest.digest();
String hashText = DatatypeConverter.printHexBinary(digest);
System.out.println(hashText);

Note: the MessageDigest is not thread-safe. Consequently, we should use a new instance for every thread.

SHA

SHA (Secure Hash Algorithm)

SHA-256 by MessageDigest

String text = "123456";
MessageDigest messageDigest = MessageDigest.getInstance("SHA-256");
messageDigest.reset();
messageDigest.update(text.getBytes(StandardCharsets.UTF_8));
byte[] digest = messageDigest.digest();
String hashText = DatatypeConverter.printHexBinary(digest);
System.out.println(hashText);

SHA3-256 by MessageDigest (since Java 9)

String text = "123456";
MessageDigest messageDigest = MessageDigest.getInstance("SHA3-256");
messageDigest.reset();
messageDigest.update(text.getBytes(StandardCharsets.UTF_8));
byte[] digest = messageDigest.digest();
String hashText = DatatypeConverter.printHexBinary(digest);
System.out.println(hashText);

Conclusion

  • SHA256 is difficult to handle than MD5 because of its size.
  • SHA256 is less secure than MD5
  • MD5 result in an output of 128 bits whereas SHA256 result output of 256 bits.

Concluding all points, it will be better to use MDA5 if you want to secure your files otherwise you can use SHA256.

Symmetric Encryption Algorithms

DES

DES (data encryption standard, 1976)

DES is Not Secure.

3DES

3DES is Not Secure.

AES

AES (advanced encryption system, 2001)

The AES algorithm has six modes of operation:

  1. ECB (Electronic Code Book) Not Recommend
  2. CBC (Cipher Block Chaining)
  3. CFB (Cipher FeedBack)
  4. OFB (Output FeedBack)
  5. CTR (Counter)
  6. GCM (Galois/Counter Mode)

Don’t use AES Electronic codebook (ECB) Mode. The AES ECB mode, or AES/ECB/PKCS5Padding (in Java) is not semantically secure.

AES encryption best practice: Don’t reuse IV with the same key.

IV

The IV (initial value or initial vector), it is random bytes, typically 12 bytes or 16 bytes. In Java, we can use SecureRandom to generate the random IV.

public static IvParameterSpec generateIv() {
byte[] iv = new byte[16];
new SecureRandom().nextBytes(iv);
return new IvParameterSpec(iv);
}

secret key

The AES secret key, either AES-128 or AES-256. In Java, we can use KeyGenerator to generate the AES secret key.

public static SecretKey getAESKey() throws NoSuchAlgorithmException {
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256, SecureRandom.getInstanceStrong());
return keyGen.generateKey();
}

The AES secret key that derived from a given password. In Java, we can use the SecretKeyFactory and PBKDF2WithHmacSHA256 to generate an AES key from a given password.

public static SecretKey getAESKeyFromPassword(char[] password, byte[] salt)
throws NoSuchAlgorithmException, InvalidKeySpecException {
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
// iterationCount = 65536
// keyLength = 256
KeySpec spec = new PBEKeySpec(password, salt, 65536, 256);
SecretKey secret = new SecretKeySpec(factory.generateSecret(spec).getEncoded(), "AES");
return secret;
}

salt

We use salt to protect rainbow attacks, and it is also a random byte, we can use the SecureRandom to generate it.

import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.PBEKeySpec;
import javax.crypto.spec.SecretKeySpec;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.KeySpec;
import java.util.ArrayList;
import java.util.List;

public class CryptoUtils {

public static byte[] getRandomNonce(int numBytes) {
byte[] nonce = new byte[numBytes];
new SecureRandom().nextBytes(nonce);
return nonce;
}

// AES secret key
public static SecretKey getAESKey(int keysize) throws NoSuchAlgorithmException {
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(keysize, SecureRandom.getInstanceStrong());
return keyGen.generateKey();
}

// Password derived AES 256 bits secret key
public static SecretKey getAESKeyFromPassword(char[] password, byte[] salt)
throws NoSuchAlgorithmException, InvalidKeySpecException {
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
// iterationCount = 65536
// keyLength = 256
KeySpec spec = new PBEKeySpec(password, salt, 65536, 256);
SecretKey secret = new SecretKeySpec(factory.generateSecret(spec).getEncoded(), "AES");
return secret;
}

// hex representation
public static String hex(byte[] bytes) {
StringBuilder result = new StringBuilder();
for (byte b : bytes) {
result.append(String.format("%02x", b));
}
return result.toString();
}

// print hex with block size split
public static String hexWithBlockSize(byte[] bytes, int blockSize) {
String hex = hex(bytes);
// one hex = 2 chars
blockSize = blockSize * 2;
// better idea how to print this?
List<String> result = new ArrayList<>();
int index = 0;
while (index < hex.length()) {
result.add(hex.substring(index, Math.min(index + blockSize, hex.length())));
index += blockSize;
}
return result.toString();
}
}

CBC Mode

Input

  • algorithm: AES/CBC/PKCS5Padding
  • iv bytes
  • aes key bits
  • secretKey: generate by password
  • Initialization Vector (IV): random bytes

GCM Mode


Asymmetric Encryption Algorithms

RSA

RSA (Rivest-Shamir-Adleman)

public static void getKeyPair() throws NoSuchAlgorithmException {
KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
kpg.initialize(2048);
KeyPair keyPair = kpg.generateKeyPair();
Key pub = keyPair.getPublic();
Key pvt = keyPair.getPrivate();
System.out.println("public key");
System.out.println(Base64.getEncoder().encodeToString(pub.getEncoded()));
System.out.println("private key");
System.out.println(Base64.getEncoder().encodeToString(pvt.getEncoded()));
}

public static String decrypt(String privateKeyBase64String, String cipherText) throws NoSuchAlgorithmException, InvalidKeySpecException, NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException {
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
EncodedKeySpec pkcs8KeySpec = new PKCS8EncodedKeySpec(Base64.getDecoder().decode(privateKeyBase64String.getBytes()));
PrivateKey privateKey = keyFactory.generatePrivate(pkcs8KeySpec);
Cipher cipher = Cipher.getInstance(keyFactory.getAlgorithm());
cipher.init(Cipher.DECRYPT_MODE, privateKey);
byte[] bytes = cipher.doFinal(Base64.getDecoder().decode(cipherText));
return new String(bytes);
}

public static String encrypt(String publicKeyBase64String, String plainText) throws NoSuchAlgorithmException, InvalidKeySpecException, NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException {
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
EncodedKeySpec pkcs8KeySpec = new X509EncodedKeySpec(Base64.getDecoder().decode(publicKeyBase64String.getBytes()));
PublicKey publicKey = keyFactory.generatePublic(pkcs8KeySpec);
Cipher cipher = Cipher.getInstance(keyFactory.getAlgorithm());
cipher.init(Cipher.ENCRYPT_MODE, publicKey);
byte[] bytes = cipher.doFinal(plainText.getBytes());
return new String(Base64.getEncoder().encode(bytes));
}

ECC

ECC (Elliptic Curve Cryptography)


References

[1] Java AES encryption and decryption

Query

QueryWrapper

QueryWrapper

// usage 1
QueryWrapper<SysUser> queryWrapper = new QueryWrapper<SysUser>()
.eq("user_id", userId)
.like(name != null, "name", name);

// usage 2
QueryWrapper<SysUser> queryWrapper = Wrappers.query(sysUser)
.eq("user_id", userId);

LambdaQueryWrapper

// usage 1
LambdaQueryWrapper<SysUser> lambdaQueryWrapper = new LambdaQueryWrapper<SysUser>()
.eq(SysUser::getUserId, userId)
.like(name != null, SysUser::getName, name);

// usage 2
LambdaQueryWrapper<SysUser> lambdaQueryWrapper = Wrappers.lambdaQuery(sysUser)
.eq(SysUser::getUserId, userId);

// usage 3
LambdaQueryWrapper<SysUser> lambdaQueryWrapper = Wrappers.lambdaQuery(SysUser.class)
.eq(SysUser::getUserId, userId);

Convert QueryWrapper to LambdaQueryWrapper

LambdaQueryWrapper<SysUser> lambdaQueryWrapper = new QueryWrapper<SysUser>()
.eq("user_id", userId)
.lambda()
.eq(SysUser::getType, type);

Basic Query

An example

List<SysUser> sysUserList = userService.list(
new QueryWrapper<SysUser>()
.select("name")
.eq("type", type)
.orderByDesc("id")
.last("limit 10"));

from

new QueryWrapper<Entity>()

select

wrapper.select("column 1", "column 2")

where

wrapper.eq("type", 1)
.like("title", "test")
.in("user_id", userIds)
.ge("age", 18)
.apply("find_in_set({0}, type)", type);
.apply("date(gmt_create)=date({0})", new Date())
.apply("date_format(gmt_modified,'%Y-%m-%d') = date({0})", new Date())

and…or

// where type = 1 or type = 2
wrapper.and(wrapper -> wrapper.eq("type", 1)
.or().eq("type", 2))
// where status = 0 and (type = 1 or (type > 10 and type < 20))
wrapper.eq("status", 0)
.and(wrapper -> wrapper.eq("type", 1)
.or(wrapper2 ->
wrapper2.gt("type", 10)
.lt("type", 20));

order by

wrapper.orderByDesc("id")

limit

wrapper.last("limit 10")

Query Conditions

find_in_set

wrapper.apply("find_in_set({0}, type)", type);

Date functions

date equal

wrapper.apply("date(gmt_create)=date({0})", new Date())
.apply("date_format(gmt_modified,'%Y-%m-%d') = date({0})", new Date())

Recent 7 days

wrapper.apply("create_time > DATE_SUB(NOW(), INTERVAL 7 DAY)"));

Page Query

  1. Request query string parameters

http://xxx.xxx/?current=1&size=10&orders[0].column=pubtime&orders[0].asc=true

  • current
  • size
  • orders[].column
  • orders[].asc
  1. Page Entity

com.baomidou.mybatisplus.extension.plugins.pagination.Page

Fields for Request parameters

  • long size
  • long current
  • List<OrderItem> orders
    • String column
    • boolean asc

Fields for response

  • List<T> records
  • long total
  1. Page Query Methods

MyBatis Plus page query method

IPage<T> selectPage(IPage<T> page, Wrapper<T> queryWrapper)

Custom page query in Mapper XML

IPage<MyResult> myPageQuery(@Param("page") Page page, @Param("param") MyParam param);
  • Pass com.baomidou.mybatisplus.extension.plugins.pagination.Page object as a parameter.
  • Using the IPage class as the return type.
  • Don’t need to add limit start, size in mapper XML. The SQL is query all rows. But MyBatis Plus automatically add limit at the end of SQL. If you want to query all, update to List<MyResult> queryAll(@Param("param") MyParam param);

One Column Query

List<Object> names = baseMapper.selectObjs(
new LambdaQueryWrapper<User>()
.select(User::getName)
.eq(User::getType, 1));
if (names == null) {
return Collections.emptySet();
} else {
return names.stream()
.map(Object::toString)
.collect(Collectors.toSet());
}

Aggregation Query

Aggregation Query methods

  • selectMaps()
List<Map<String, Object>> mapList = userMapper.selectMaps(
new QueryWrapper<SysUser>()
.select("type as name, count(*) as typeCount")
.groupBy("type"));
  • selectObjs()
List<Object> mapList = userMapper.selectObjs(
new QueryWrapper<SysUser>()
.select("sum(num) as typeTotal")
.groupBy("type"));
  • selectCount()
userMapper.selectCount(queryWrapper);

select

queryWrapper.select("count(*) as typeCount");
queryWrapper.select("submit_content_type as name, count(*) as value")

group by

queryWrapper.groupBy("type"));

having

queryWrapper.having("COUNT(*) > 10"))

Others

Non-query fields

Use @TableField(exist = false)

@TableField(exist = false)
private IdName creator;

Use @TableName(excludeProperty={})

@TableName(value = "my_table_name", excludeProperty = {"creator"})
public class User {
private IdName creator;
}

Using @TableField(condition = SqlCondition.LIKE)

@TableField(condition = SqlCondition.LIKE)  
private String name;

Using @TableField(whereStrategy = FieldStrategy.NOT_EMPTY)

  • IGNORED: 不判断
  • NOT_NULL: 非NULL判断
  • NOT_EMPTY: 非空判断

Using MySQL keyword as a column name

@TableField(value = "`type`")
private Integer type;

Using another entity name

@Alias("UserV2")
public class User {
}

DML

update it to null when it value is null

@TableField(fill = FieldFill.UPDATE)
private String name;

logical delete

// Note that the field for logical deletion cannot be modified by update() method
@TableLogic(value="0",delval="1")
private Boolean delFlag;

Update

userService.update(new UpdateWrapper<User>()
.set("name", "updateName")
.eq("id", 1));

MyBatis Plus Configurations

mybatis-plus:
mapper-locations: classpath:mybatis/mapper/*.xml
type-aliases-package: com.example.entity,com.example.modules.**.entity
global-config:
db-config:
where-strategy: not_empty
logic-delete-field: deleteFlag
logic-delete-value: 1
logic-not-delete-value: 0

More configurations reference MyBatis Plus使用配置

Background

Running the Spring Boot project occurs an error.

Error Info

Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'spring.datasource.druid.initialSize' in value "${spring.datasource.druid.initialSize}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:178)
... 97 common frames omitted

Solutions

1. Check that the property you need exists in a configuration file. Find out in which configuration file your injected property is.

2. Check whether the property spring.profiles.active value is correct in application.yml. Make sure the active profile is in your project.

3. Check whether your injected property exists in application.yml (or application.properties) and the active profile application-xxx.yml (or application-xxx.properties).

Reasons

The value of the property spring.profiles.active in my application.yml is spring.profiles.active=pro, but I don’t have the application-pro.yml file. The property’s value spring.profiles.active=pro should update to prod.

Background

When I do an insert operation to save the item data to MySQL, occurs an error.

Error Info

Error updating database.  Cause: java.sql.SQLException: Data truncated for column 'xxx' at row 1

Solutions

Check whether the column type is right.

Check whether the length of the field value over the length of the column.

Reasons

The length of the field value over the length of the column.

My passed field value is a set type value, but the column type is enum. The column type should update to set.

0%