Taogen's Blog

Today I try to delete a directory with 485,000+ files on Linux server. It shows the error message below.

/bin/rm: Argument list too long.

If you’re trying to delete a very large number of files in a single directory at one time, you will probably run into this error. The problem is that when you type something like rm -f /data/*. So how do we working with directories with a large number of files? Let talk about it below.

Show first files

To show first n files in the directory instead of to show all files.

ls -U | head -10

Count the number of files

ls -f | wc -l

Remove files

Using for Loop

for f in *.pdf; do echo rm "$f"; done
for f in *.pdf; do rm "$f"; done

Using Find

Using find -delete. (2000 files/second, it is about three time faster rm)

find . -maxdepth 1 -type f -name "$.split-*" -print -delete

Using find and xargs. To find every file and pass them one-by-one to the “rm” command (not recommend, xargs is dangerous*)

find . -type f -exec rm -v {} \;

find . -maxdepth 1 -name "$.split-*" -print0 | xargs -0 rm

View and Edit large files

Open the 4.14 GB sql file

Computer

• ThinkPad T14s
• Power mode: Best performance
• Processor: 10th Generation Intel® Core™ i5-10210U Processor (1.60 GHz, up to 4.20 GHz with Turbo Boost, 4 Cores, 8 Threads, 6 MB Cache)
• Memory: 16GB
• Hard Drive: 512 GB PCIe SSD

Result

• vim: opened about 13 seconds. saved about 20 seconds.
• sublime: opened 3 min 30 seconds. saved about 13 seconds.
• less: opened instantly. can’t edit.
• Intellij IDEA: opened about 3 seconds. can’t edit. and tip “The file is too large. Read-only mode.”
• emacs: opened about 15 seconds. saving crashed.
• Nodepad++: Can’t open. Error: file is too big opend by Notepad++.
• VS code: Can’t open. open crashed

Best overall: vim.

Best for view: less or Intellij IDEA.

Export data from or Import data to remote MySQL server

Export 4.14 GB data from remote MySQL server

Cost time: 3 hours.

best exporting data at night.

软件开发并不只是写代码。一次提交的代码可能很少，但可能也需要做大量其它相关的工作。写代码的时间只是软件开发工作的一部分，还要很多其它事情要做，如：分析和理解需求，设计系统和功能，尝试多种解决方案后找到合适的解决方案并实现代码，与其他同事进行对接和沟通，测试代码和修复bug等。

我认为要控制好软件开发的进度，需要做好以下这两件事：

1. 时间规划和任务划分。正确地估算，设置合理的预期时间，调整开发时间安排。这个需要比较丰富的开发经验才能很好地估算。
2. 更高效、稳定地完成任务。这个需要平时的积累。

接下来，将针对第2点“如何更高效、稳定地完成任务”进行详细地描述。

如何更高效、稳定地开发

需求分析

1. 明确需求。有需求规格说明书文档。有文档方便快速回顾。
2. 明确 UI 或原型。
3. 对需求的理解要清晰、明确，有问题及时沟通。需求是软件开发的目标，目标不清楚，只会浪费时间做无用功。这是非常重要关键的，不要不好意思去问或者拖延不去问。

系统设计和详细设计

1. 编写系统设计文档，详细设计文档。有文档方便快速回顾。
2. 平时可以积累一些设计的模板。系统设计无非是架构设计、数据库设计、API 设计等。按照规范去做，刚开始可能会很慢，多做几次就好一些。
3. 做好了软件设计，心里会比较有底，能够大致知道有哪些事情要做和自己所处的进度。知道了自己的进度，就能很好地进行调整，而不是前面很松懈，后面一直加班追赶进度。

代码实现

1. 平时积累学习一些相关的技术。防止开发过程中，边学习边开发，影响进度。
2. 平时积累一些代码。通用工具代码、功能实现代码。可以更快、更省力地写代码。
3. 平时积累一些代码问题解决的笔记或博客。遇到相似问题可以有参考，防止卡住进度。
4. 编写整洁、易读、方便扩展的代码。方便快速地修改、扩展代码，以及减少潜在问题。
5. 快速实现，不断迭代。初期不要过分追求完美的代码。“过早优化是万恶之源”。

测试

1. 写好测试用例文档，方便进行完整、严格的测试。平时可以积累一些常见功能的测试用例。
2. 提交代码前，通过测试用例，减少修复 bug 的时间。
3. 修改代码，做一些回归测试，减少修复 bug 的时间。
4. 写单元测试或者使用测试驱动开发，可以更自信地重构和优化代码，防止重构时引入新的 bug。

Iconfont 是阿里妈妈MUX倾力打造的矢量图标管理、交流平台。

设计师将图标上传到 iconfont 平台，用户可以自定义下载多种格式的icon，平台也可将图标转换为字体，便于前端工程师自由调整与调用。

Unicode 引用

• 支持按字体的方式去动态调整图标大小，颜色等等。
• 默认情况下不支持多色，直接添加多色图标会自动去色。

第一步：引入项目的样式文件

找到你的项目：资源管理 –> 我的项目 –> 我参与的项目 –> xxx项目

选择使用 Unicode 外链或者将代码下载至本地

/* 引入外链 */
@font-face {
  font-family: 'myIcon'; /*自定义，默认为项目名称*/
  src: url('//at.alicdn.com/t/c/font_xxx.woff2?t=1686623243724') format('woff2'),
       url('//at.alicdn.com/t/c/font_xxx.woff?t=1686623243724') format('woff'),
       url('//at.alicdn.com/t/c/font_xxx.ttf?t=1686623243724') format('truetype');
}

/* 引入本地文件 */
@font-face {
  font-family: 'myIcon'; /*自定义，默认为项目名称*/
  src: url('iconfont.woff2?t=1625109690491') format('woff2'),
       url('iconfont.woff?t=1625109690491') format('woff'),
       url('iconfont.ttf?t=1625109690491') format('truetype');
}

第二步：定义全局的 unicode 图标样式

/* 自定义 className */
.myIconFont {
  font-family: "myIcon" !important; /* 需要与你的@font-face中的font-family保持一致 */
  font-size: 16px;
  font-style: normal;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

第三步：挑选相应图标并获取字体编码，应用于页面

<span class="myIconFont">&#x33;</span>
<span class="myIconFont">&#xe718;</span>
<i class="myIconFont">&#xe718;</i>

修改图标大小：设置 font-size

<span class="myIconFont" style="font-size: 500px;">&#x33;</span>

修改图标颜色：设置 color

<span class="myIconFont" style="color: #8BC34A;">&#x33;</span>

Font Class 引用

font-class 是 Unicode 使用方式的一种变种，主要是解决 Unicode 书写不直观，语意不明确的问题。

与 Unicode 使用方式相比，具有如下特点：

• 相比于 Unicode 语意明确，书写更直观。可以很容易分辨这个 icon 是什么。
• 因为使用 class 来定义图标，所以当要替换图标时，只需要修改 class 里面的 Unicode 引用。

第一步：引入项目的样式文件

找到你的项目：资源管理 –> 我的项目 –> 我参与的项目 –> xxx项目

选择使用 Font class 外链或者将代码下载至本地

<!-- 引入外链 -->
<link rel="stylesheet" type="text/css" href="//at.alicdn.com/t/font_xxx.css">

<!-- 引入本地文件 -->
<link rel="stylesheet" type="text/css" href="./iconfont.css">

第二步：挑选相应图标并获取类名，应用于页面

<!-- 注意：第一个 class 需要与 iconfont.css 文件中的 @font-face {font-family: "myIconFont"} 保持一致，默认项目名，可自定义。第二个 class 是icon+图标的名称 -->
<span class="myIconFont icon-xxx"></span>

改变图标的大小：重写 font-size 属性

<span class="myIconFont icon-xxx" style="font-size: 500px;"></span>

Symbol 引用

这是一种全新的使用方式，应该说这才是未来的主流，也是平台目前推荐的用法。这种用法其实是做了一个 SVG 的集合，与另外两种相比具有如下特点：

• 支持多色图标了，不再受单色限制。
• 通过一些技巧，支持像字体那样，通过 font-size, color 来调整样式。
• 兼容性较差，支持 IE9+，及现代浏览器。
• 浏览器渲染 SVG 的性能一般，还不如 png。

第一步：引入项目的样式文件

找到你的项目：资源管理 –> 我的项目 –> 我参与的项目 –> xxx项目

选择使用 Symbol 外链或者将代码下载至本地

<!-- 引入外链 -->
<script type="text/javascript" src="//at.alicdn.com/t/c/font_xxx.js">

<!-- 引入本地文件 -->
<script type="text/javascript" src="./iconfont.js"></script>

第二步：定义全局的 symbol 图标样式

<style>
/*自定义样式class名称*/
.myIcon {
  width: 1em;
  height: 1em;
  vertical-align: -0.15em;
  fill: currentColor;
  overflow: hidden;
}
</style>

第三步：挑选相应图标并获取类名，应用于页面

<!-- 样式名称与上一步定义的保持一致 -->
<svg class="myIcon" aria-hidden="true">
  <use xlink:href="#icon-xxx"></use>
</svg>

改变图标的大小：设置 font-size 属性

<svg class="myIcon" aria-hidden="true" style="font-size: 500px;">
  <use xlink:href="#icon-xxx"></use>
</svg>

改变图标的大小：同时设置 width 和 height 属性（只设置一个没有作用）

<svg class="myIcon" aria-hidden="true" style="width: 300px; height: 300px;">
  <use xlink:href="#icon-xxx"></use>
</svg>

References

• @font-face - MDN
• iconfont 使用帮助

Background

Using HTTP domain visit my website is ok, but HTTPS not.

Error Info

ERR_TIMED_OUT

This site can’t be reached
xxx.xxx.com took too long to respond.
Try:
- Checking the connection
- Checking the proxy and the firewall
- Running Windows Network Diagnostics
ERR_TIMED_OUT

Solutions

ERR_TIMED_OUT represents this is a connection error, not SSL certificate errors.

Common Solution

• Check whether the server IP or HTTP domain can be visited. If ok, the connection is OK.
• Check whether the server reversed proxy is right, and server firewall is open 80 and 443.
• Check whether the client browser and client network is right.

Current HTTP is work represented the connection is ok. And client browser can visit other HTTPS websites, represented client is ok.

You need to check your server reversed proxy is right, and ports 80 and 443 of your server firewall are open .

Reasons

My server’s port 443 is not open.

References

[1] https timeout while http works

Background

I call APIs of OSS (object storage services) to upload my local file with Java Input Stream.

Error Info

The uploaded file is 0 bytes.

Solutions

• Check your accessed file exists and the file is not 0 byte.
• Before the inputStream upload, using Java InputStream available() method to check out whether the remaining number of bytes that can be read from the file input stream is 0.
• Make sure the remaining number of bytes that can be read from the file input stream is right.

Reasons

The reason of the size of my uploaded file is 0 byte is I read the input stream two times.

Most of input stream classes in Java API are not support read more than once. Because some Input Stream classes of Java API not support mark() and reset() method.

Although the FilterInputStream child classes can support read more than once by its mark(), reset() methods, and internal cache array byte buf[], you still need to call it manually. The reset() method is for repositions this stream to the position at the time the mark method was last called on this input stream.

Receive from Request URL Query String

Integer Array

Frontend Pass with HTTP request

• url?ids=1,2,3
• url?ids=1&ids=2&ids=3

Backend receive in controller methods

• Integer[] ids
• @RequestParam Integer[] ids
• @RequestParam List<Integer> ids

String Array

Frontend Pass with HTTP request

• url?ids=a,b,c
• url?ids=a&ids=b&ids=c

Backend receive in controller methods

• String[] ids
• @RequestParam String[] ids
• @RequestParam List<String> ids

Receive from Form Data

Integer Array

Frontend Pass with HTTP request

• ids=1,2,3
• ids=1, ids=2, ids=3

Backend receive in controller methods

• Integer[] ids
• @RequestParam Integer[] ids
• @RequestParam List<Integer> ids

String Array

Frontend Pass with HTTP request

• ids=a,b,c
• ids=a, ids=b, ids=c

Backend receive in controller methods

• String[] ids
• @RequestParam String[] ids
• @RequestParam List<String> ids

Receive from request body JSON

Integer Array

Frontend Pass with HTTP request

• [1,2,3]

Backend receive in controller methods

• @RequestBody Integer[] ids
• @RequestBody List<Integer> ids

String Array

Frontend Pass with HTTP request

• [“a”,”b”,”c”]

Backend receive in controller methods

• @RequestBody String[] ids
• @RequestBody List<String> ids

Download file streams with Axios

download.js

const axios = require('axios').default; // or import axios from 'axios'
const baseUrl = process.env.VUE_APP_BASE_API

/**
 * Download file stream
 *
 * @param uri e.g. /web/article/exportExcel
 * @param params e.g. {id: 1}
 */
export function download(uri, params) {
    var url = baseUrl + uri
    return axios.get(url, {
        params:params,
        responseType: 'blob',
    }).then((response) => {
        // response.data: file stream or error message defined by developers such as {"msg":"something went wrong...","code":500}
        resolveBlob(response);
    }).catch(error => {
        // error.response.data: spring framework ResponseEntity object
        alert("Fail to download file")
    })
}

function resolveBlob(response) {
    const headerval = response.headers['content-disposition'];
    if (headerval != null) {
        let filename = headerval.split(';')[1].split('=')[1].replace('"', '').replace('"', '');
        filename = decodeURI(filename);
        const url = window.URL.createObjectURL(new Blob([response.data]));
        const link = document.createElement('a');
        link.href = url;
        link.setAttribute('download', filename);
        document.body.appendChild(link);
        link.click();
        window.URL.revokeObjectURL(url);
        link.remove();
    } else {
        handleKnownException(response);
    }
}

function handleKnownException(response) {
    var reader = new FileReader();
    reader.onload = function() {
        if (reader.result != null) {
            const responseData = JSON.parse(reader.result);
            if (responseData.code == 500) {
                alert(responseData.msg);
            }
        }
    }
    reader.readAsText(response.data);
}

Notice:

• responseType: 'blob'
• filename = decodeURI(filename)

article.vue

<template>
...
</template>

<script>
    import { downLoadFile } from '@/utils/download';

    export default {
        ...
        methods: {
            handleExportExcel() {
                this.fullscreenLoading = true;
                downLoadFile("/web/article/exportExcel", this.searchParams)
                    .then(() => {
                    this.fullscreenLoading = false;
                });
            }
        }
    }
</script>

Here are several ways to access parent context inside child context.

1. Use ES6 Arrow functions.

An arrow function does not have its own bindings to this or super, this is inherited from the scope. In regular function, this is the function itself (it has its own scope).

getData() {
	ajaxRequest(query).then(response => {
        this.data = response.data;
    });
}

2. Store reference to context/this inside another variable, If you can’t use ES6 syntax.

getData() {
	let self = this;
	ajaxRequest(query).then(function(response) {
        self.data = response.data;
    });
}

References

[1] Arrow function expressions

[2] How to access the correct this inside a callback?

1. Put files into /public directory

2. Write your download link

Download /public/pdf/instruction.pdf

<a href="/pdf/instruction.pdf" download>download instruction</a>

Preview and download /public/pdf/instruction.pdf

<a href="/pdf/instruction.pdf" target="_blank">download instruction</a>