currying

Posted on 2021-01-15 Edited on 2024-05-30 In JavaScript

柯里化是一种将使用多个参数的一个函数转换成一系列使用一个参数的函数的技术

function add(a, b) {
    return a + b;
}

// 执行 add 函数，一次传入两个参数即可
add(1, 2) // 3

// 假设有一个 curry 函数可以做到柯里化
var addCurry = curry(add);
addCurry(1)(2) // 3

closure

Posted on 2021-01-08 Edited on 2024-05-30 In JavaScript

前置知识：JavaScript是静态作用域

闭包：访问自由变量的函数

var a = 1;//既不是foo的局部变量，也不是foo函数的参数，a为自由变量

function foo() {
    console.log(a);
}

foo();//1

即使上下文被销毁，它仍然存在，因为在作用域链上被引用了，是js的一个特性，目前如PHP，Java不会原生支持

面试题

常见的新手面试题，我遇到过好几次（作用域+闭包考点）

var data = [];

for (var i = 0; i < 3; i++) {
  data[i] = function () {
    console.log(i);
  };
}

data[0]();
data[1]();
data[2]();


//closure 
var data = [];

for (var i = 0; i < 3; i++) {
  data[i] = (function (i) {
        return function(){
            console.log(i);
        }
  })(i);
}

data[0]();//不用找global的i
data[1]();
data[2]();

vscode访问服务器文件

Posted on 2021-01-05 Edited on 2024-05-30 In skills

1.install remote ssh in vscode

2.click remote explorer and select ssh targets

3.click remote ssh configure or press F1 and input remote-ssh:Open configuration file

4.selete path ~/.ssh/config,and modify config file

if you dont have rsa ,please generate keys before

//optional
ssh-keygen
# passphrase can be empty and then generate keys in `~/.ssh`
# put *.pub (public key) to your server (~/.ssh/) and excute `cat id_rsa.pub >> authorized_keys` to merge Previous file
# now rsa keys are ready

Host alias
    HostName 8.888.88.8
    User root
    IdentityFile ~/.ssh/id_rsa
    RSAAuthentication yes
    PubkeyAuthentication yes
    PasswordAuthentication no

Host alias–>your remote server name
hostName–>server ip
User–>login username
IdentityFile–>private key path
RSAAuthentication–>optional
PubkeyAuthentication–>optional
PasswordAuthentication–>no password login

5.login without password ready

2020-10-16-promise用法

Posted on 2020-10-16 Edited on 2024-05-30

What

ECMAscript 6 原生提供了 Promise 对象。

Promise 对象代表了未来将要发生的事件，用来传递异步操作的消息。

Promise 对象有以下两个特点:

1、对象的状态不受外界影响。Promise 对象代表一个异步操作，有三种状态：

pending: 初始状态，不是成功或失败状态。
fulfilled: 意味着操作成功完成。
rejected: 意味着操作失败。

只有异步操作的结果，可以决定当前是哪一种状态，任何其他操作都无法改变这个状态。这也是 Promise 这个名字的由来，它的英语意思就是「承诺」，表示其他手段无法改变。

2、一旦状态改变，就不会再变，任何时候都可以得到这个结果。Promise 对象的状态改变，只有两种可能：从 Pending 变为 Resolved 和从 Pending 变为 Rejected。只要这两种情况发生，状态就凝固了，不会再变了，会一直保持这个结果。就算改变已经发生了，你再对 Promise 对象添加回调函数，也会立即得到这个结果。这与事件（Event）完全不同，事件的特点是，如果你错过了它，再去监听，是得不到结果的。

var promise = new Promise(function(resolve, reject) {
    // 异步处理
    // 处理结束后、调用resolve 或 reject
});

以上来自:菜鸟https://www.runoob.com/w3cnote/javascript-promise-object.html

Why

因为在2020年01月07日有一篇文章讲了使用promise实现延时队列的一道面试题，因为之前写业务没有用到过所以一直以为用处不大，但今天对接阿里的录音文件识别转文字的接口中，示例代码是一个setInterval轮询得到结果的一种方式，但是他带来了一个很严重的问题

！！没有办法返回前端转文字的结果！！

大概代码如下

//url:https://help.aliyun.com/document_detail/94242.html?spm=a2c4g.11174283.6.601.15eb7275a8rq00
// 这段代码会异步执行，可以得到结果，但是直接用这个代码返回给前端
client.submitTask(taskParams, options).then((response) => {
        console.log(response);
        // 服务端响应信息的状态描述StatusText。
        var statusText = response.StatusText;
        if (statusText != 'SUCCESS') {
            console.log('录音文件识别请求响应失败!')
            return;
        }
        console.log('录音文件识别请求响应成功!');
        // 获取录音文件识别请求任务的TaskId，以供识别结果查询使用。
        var taskId = response.TaskId;
        /**
         * 以TaskId为查询参数，提交识别结果查询请求。
         * 以轮询的方式进行识别结果的查询，直到服务端返回的状态描述为"SUCCESS"、SUCCESS_WITH_NO_VALID_FRAGMENT，
         * 或者为错误描述，则结束轮询。
        */
        var taskIdParams = {
            TaskId : taskId
        };
        var timer = setInterval(() => {
            client.getTaskResult(taskIdParams).then((response) => {
                console.log('识别结果查询响应：');
                console.log(response);
                var statusText = response.StatusText;
                if (statusText == 'RUNNING' || statusText == 'QUEUEING') {
                    // 继续轮询，注意间隔周期。
                }
                else {
                    if (statusText == 'SUCCESS' || statusText == 'SUCCESS_WITH_NO_VALID_FRAGMENT') {
                        console.log('录音文件识别成功：');
                        var sentences = response.Result;
                        console.log(sentences);
                    }
                    else {
                        console.log('录音文件识别失败!');
                    }
                    // 退出轮询
                    clearInterval(timer);
                }
            }).catch((error) => {
                console.error(error);
                // 异常情况，退出轮询。
                clearInterval(timer);
            });
        }, 10000);
    }).catch((error) => {
        console.error(error);
    });
}

How

使用promise进行包裹，等到promise内部的函数取到了结果在返回

if (statusText == 'SUCCESS' || statusText == 'SUCCESS_WITH_NO_VALID_FRAGMENT') {
                        console.log('录音文件识别成功：');
                        var sentences = response.Result;
                        console.log(sentences);
  											//这里新增resolve
} else {
                        console.log('录音文件识别失败!');
}

外层通过如下代码实现

var promise = new Promise(function(resolve, reject) {
    // 异步处理
    // 处理结束后、调用resolve 或 reject
});

async function getWords() {
      return new Promise((resolve, reject) => {
        client
          .submitTask(taskParams, options)
          .then(response => {
            console.log(response);
            // 服务端响应信息的状态描述StatusText。
            const statusText = response.StatusText;
            if (statusText != 'SUCCESS') {
              console.log('录音文件识别请求响应失败!');
            }
            console.log('录音文件识别请求响应成功!');
            // 获取录音文件识别请求任务的TaskId，以供识别结果查询使用。
            const taskId = response.TaskId;
            /**
             * 以TaskId为查询参数，提交识别结果查询请求。
             * 以轮询的方式进行识别结果的查询，直到服务端返回的状态描述为"SUCCESS"、SUCCESS_WITH_NO_VALID_FRAGMENT，
             * 或者为错误描述，则结束轮询。
             */
            const taskIdParams = {
              TaskId: taskId,
            };

            const timer = setInterval(() => {
              client
                .getTaskResult(taskIdParams)
                .then(response => {
                  console.log('识别结果查询响应：');
                  console.log(response);
                  const statusText = response.StatusText;
                  if (statusText == 'RUNNING' || statusText == 'QUEUEING') {
                    // 继续轮询，注意间隔周期。
                  } else {
                    if (
                      statusText == 'SUCCESS' ||
                      statusText == 'SUCCESS_WITH_NO_VALID_FRAGMENT'
                    ) {
                      console.log('录音文件识别成功：');
                      let sentences = '';
                      for (const s of response.Result.Sentences) {
                        sentences += s.Text;
                      }
                      console.log(response.Result);
                      resolve(sentences);//**重点**//
                      // return sentences;
                    } else {
                      console.log('录音文件识别失败!');
                    }
                    // 退出轮询
                    clearInterval(timer);
                  }
                })
                .catch(error => {
                  console.error(error);
                  // 异常情况，退出轮询。
                  clearInterval(timer);
                });
            }, 10000);
          })
          .catch(error => {
            console.error(error);
          });
      });
    }
    return await getWords();//返回前端，翻译结果

另外记录一件事情，左侧单元图标地址：https://fontawesome.com/v4.7.0/icons/

NodeJS

Posted on 2020-09-27 Edited on 2024-05-30

https://github.com/theanarkh/understand-nodejs

文档还是不错的

Jupyter

Posted on 2020-09-25 Edited on 2024-05-30

what

Jupiter = Julia + Python + R

Jupyter notebook（http://jupyter.org/）是一种 Web 应用，能让用户将说明文本、数学方程、代码和可视化内容全部组合到一个易于共享的文档中。

why

将代码和文档结合在一起，更直观的编写人工智能，大数据的代码
分块运行
直接运行shell不需要切换环境
so on

how

Download images

1 2	$docker pull jupyter jupyter/scipy-notebook:latest $docker run -itd --rm -p 1000:8888 -e JUPYTER_ENABLE_LAB=yes -v /home/zehai/jupyter:/home/jovyan/work --name jupyter jupyter/scipy-notebook:latest

docker logs -f container’s ID and find token

To access the notebook, open this file in a browser:
    file:///home/jovyan/.local/share/jupyter/runtime/nbserver-6-open.html
Or copy and paste one of these URLs:
    http://896bb1e66101:8888/?token=fda8565a9b5cd5b8c621b45322ee72f716fd7ddea089fb51
 or http://127.0.0.1:8888/?token=fda8565a9b5cd5b8c621b45322ee72f716fd7ddea089fb51

more info visit official docs
enjoy (pics powered by cherevero)

chevereto

Posted on 2020-09-15 Edited on 2024-05-30 In Chevereto

what

To solve some problems

some web only use markdown and can’t upload pictures,such as v2ex.com
some pics you don’t want to give it to others for long time,such as your interesting story
give your blog’s can speed when download bigger pics
and so on

Picture Bed can offer you a excellent platform to share your pictures and protect them, however it has a problem that you need a server to run the service,even though you can use 七牛云,alioss,weibo for free.

why

Chevereto is aim what I find

dockerhub has chevereto images
Combined with ShareX (only for windows😢),chevereto can write markdown essay easily
it has api ,you can make it stronger
Chevereto Free v1.2.2 now
Something others you can discover by yourself

how

Chevereto is a php project , I use docker to run it

docker pull nmtan/chevereto:latest

//use docker-compose.yml(next block)

// or docker run 
docker run -it --name chevereto -d     -p 8000:80     -v "/home/xxx/images":/var/www/html/images     -e "CHEVERETO_DB_HOST=127.0.0.1"     -e "CHEVERETO_DB_USERNAME=root"     -e "CHEVERETO_DB_PASSWORD=rootpass"     -e "CHEVERETO_DB_NAME=chevereto"     -e "CHEVERETO_DB_PREFIX=chv_"  nmtan/chevereto
//-v save photos in server instead of container
//-e mysql:5.7.31 host,username,password,db_name(db must exist first)
//open chrome and input 127.0.0.1:8000

//this is docker-compose.yml
version: '3'

services:
  db:
    image: mariadb
    volumes:
      - ./database:/var/lib/mysql:rw
    restart: always
    networks:
      - private
    environment:
      MYSQL_ROOT_PASSWORD: xxxxx
      MYSQL_DATABASE: xxxxx
      MYSQL_USER: xxxxx
      MYSQL_PASSWORD: xxxxx

  chevereto:
    depends_on:
      - db
    image: nmtan/chevereto
    restart: always
    networks:
      - private
    environment:
      CHEVERETO_DB_HOST: db
      CHEVERETO_DB_USERNAME: xxxxxx
      CHEVERETO_DB_PASSWORD: xxxxx
      CHEVERETO_DB_NAME: xxxxx
      CHEVERETO_DB_PREFIX: chv_
    volumes:
      - ./images:/var/www/html/images:rw
      - ./php.ini:/usr/local/etc/php/php.ini:ro
    ports:
      - 8080:80

networks:
  private:


// start command
nohup docker-compose up  &> run.log &
disown

You maybe run into a stone wall when you first visit 127.0.0.1:8000

Before you can use docker exec -it chevereto bash into container /var/www/html
no permission write phots to /home/xxx/images,you can use chmod -R 777 /home/xxx/images
no permission update chevereto from 1.1.4 to1.2.2 ,no update possible: /app/install/update/temp/ path,that is no temp folder in /app/install/update/ under version 1.2.0,you can mkdir temp and then chmod -R 777 ./temp and then refresh the webpage ,the prics bed will update successfully

So , you can use ip address to visit your chevereto . However , we usually use domain name such as example.com to visit web, a https isn ecessary as well

1.Use aliyun to apply a free ssl license for a domain name such as pics.example.com
2.Download pem and keys to your server and put it in nginx conf folder
3.Use the conf as follows

server {
      listen 80;
      server_name  pics.example.com;
      return 301 https://pics.example.com$request_uri;
  }

  server {
      listen 443 ssl;
      server_name  pics.example.com;
      gzip on;    

      ssl_certificate cert/xxxxxx9_pics.example.com.pem; # pem's filename
      ssl_certificate_key cert/xxxxxx9_pics.example.com.key;# key's filename

      location / {
          proxy_redirect off;
          proxy_pass http://dockername;

          proxy_set_header  Host                $http_host;
          proxy_set_header  X-Real-IP           $remote_addr;
          proxy_set_header  X-Forwarded-Ssl     on;
          proxy_set_header  X-Forwarded-For     $proxy_add_x_forwarded_for;
          proxy_set_header  X-Forwarded-Proto   $scheme;
          proxy_set_header  X-Frame-Options     SAMEORIGIN;

          client_max_body_size        100m;
          client_body_buffer_size     128k;

          proxy_buffer_size           4k;
          proxy_buffers               4 32k;
          proxy_busy_buffers_size     64k;
          proxy_temp_file_write_size  64k;
      }
  }

1. And then you can visit https://pics.example.com

That‘s my story that building pics bed ,and hope to help you.

2020-09-28 append

use picgo，upload pictures in typora to chevereto

GitHub download picgo,mac use .dmp,and then install it
open 插件设置,search chevereto and install chevereto 1.0.0
Open 图床设置>Chevereto Uploader and put in params,

Url is your upload service ip/domain

Key is chevereto api in Dashboard>Settings>API>API v1 key

param is not in use now

1 2	Url:https://example.com/api/1/upload Key:xxx

Click 确定 and 设为默认图库
make sure server is on PicGo设置>设置Server>点击设置,if it is on ,noting should be done

and then we modify config in typaro

Open Typora and open ‘preferences>Images`
Choose Upload images in when Insert and check apply above rules to local images and apply above rules to online images in option, and I suggest you check both of them to approve all pics managed by chevereto
Choose PicGo.app in images Uploader and click Test Uploader to test your upload pictures automatically

for more information ,youcan visit

thanks

RabbitMQ

Posted on 2020-09-14 Edited on 2024-05-30 In MQ

what

MQ-message queue

三足鼎立

rocketmq -Made by Java 吞吐量高一些阿里中间件

rabbitmq -Made by Erlang

Kafka-

以后有更多的了解再补充性能/功能差距

why

功能：解耦(双方通过mq交互)、异步、削峰

应用：

阿里双11

问题：

处理好新增的复杂性
处理好系统可用性

how

之所以选择rabbitmq是因为rocketmq的nameserver所需要的内存太大了，更何况boker，对于1C2G的乞丐机器来说根本跑不起来

1.docker run

Because of rocketmq need more than

docker pull rabbitmq:management

docker run -dit --name rabbitmq -e RABBITMQ_DEFAULT_USER=admin -e RABBITMQ_DEFAULT_PASS=admin -p 15672:15672 -p 5672:5672 rabbitmq:management

--name containername
-e RABBITMQ_DEFAULT_USER 参数用户名，密码同理
-p 端口映射，主机：容器，15672-UI，5672-service
rabbitmq:management image's name

2.Usage

1.open chrome and input ‘localhost:15672’ or ‘192.168.1.1:15672’ then you can touch rabbitmq UI

Overview–the queued msg, msg rate in your server, some global counts, your nodes stats (if u use the above method,you only see one node in the screen ),you also can build a cluster with more nodes

Connections–

Channels–

Exchanges–direct,fanout,headers,match,trace,topic

Queses–

Admin–users management with passport && permission

2.use 5672 in your code

amqp.connect({
            protocol: 'amqp',
            hostname: 'example.com',//localhost
            port: '5672',
            username: 'admin',
            password: 'xxx',
            vhost: '/',//important
        })

EventLoop Source

Posted on 2020-07-24 Edited on 2024-05-30 In Node

eventLoop

之前也有过章节
node整理
 Node.js

有看到石墨技术文档

cnode技术文档，作者：youth7

记录以下知识点：

nodejs的event是基于libuv，浏览器的event loop则在html5的规范中明确定义，两个事物有明显的区别
process.nextTick()在6个阶段结束的时候都会执行

eventLoop

timers	执行`setTimeout()` 和 `setInterval()`中到期的callback
I/O callbacks	上一轮循环中有少数的I/Ocallback会被延迟到这一轮的这一阶段执行
idle, prepare	仅内部使用
poll	最为重要的阶段，执行I/O callback，在适当的条件下会阻塞在这个阶段
check	执行setImmediate的callback
close callbacks	执行close事件的callback，例如`socket.on("close",func)`

   ┌───────────────────────┐
┌─>│        timers         │
│  └──────────┬────────────┘
│  ┌──────────┴────────────┐
│  │     I/O callbacks     │
│  └──────────┬────────────┘
│  ┌──────────┴────────────┐
│  │     idle, prepare     │
│  └──────────┬────────────┘      ┌───────────────┐
│  ┌──────────┴────────────┐      │   incoming:   │
│  │         poll          │<─────┤  connections, │
│  └──────────┬────────────┘      │   data, etc.  │
│  ┌──────────┴────────────┐      └───────────────┘
│  │        check          │
│  └──────────┬────────────┘
│  ┌──────────┴────────────┐
└──┤    close callbacks    │
   └───────────────────────┘

# /deps/uv/src/unix/core.c
int uv_run(uv_loop_t* loop, uv_run_mode mode) {
  int timeout;
  int r;
  int ran_pending;

  r = uv__loop_alive(loop);
  // if(uv_has_active_hanles||uv_has_active_reqs || lop->closing_handles!=NULL)retrun true 
  if (!r)
    uv__update_time(loop);

  while (r != 0 && loop->stop_flag == 0) {
    uv__update_time(loop);
    // main
    uv__run_timers(loop);//timer phase
    ran_pending = uv__run_pending(loop);//IO callback pharse
    uv__run_idle(loop);//idle phase
    uv__run_prepare(loop);// prepare phase
    // main end

    timeout = 0;
    if ((mode == UV_RUN_ONCE && !ran_pending) || mode == UV_RUN_DEFAULT)
      timeout = uv_backend_timeout(loop);

    uv__io_poll(loop, timeout);//poll phase
    uv__run_check(loop);//check phase
    uv__run_closing_handles(loop);//closing pharse

    if (mode == UV_RUN_ONCE) {
      /* UV_RUN_ONCE implies forward progress: at least one callback must have
       * been invoked when it returns. uv__io_poll() can return without doing
       * I/O (meaning: no callbacks) when its timeout expires - which means we
       * have pending timers that satisfy the forward progress constraint.
       *
       * UV_RUN_NOWAIT makes no guarantees about progress so it's omitted from
       * the check.
       */
      // UV_RUN_ONCE 至少有一个回调执行,不然该循环就空转了，满足前进要求
      // 这也是[文章](https://zehai.info/2020/04/10/2020-04-10-eventloop/)中写到：
      // poll为空，eventloop将检查timer是否有快到的，如果需要执行，eventloop将要进入timers阶段来顺序执行timer callback
      uv__update_time(loop);
      uv__run_timers(loop);
    }

    r = uv__loop_alive(loop);
    if (mode == UV_RUN_ONCE || mode == UV_RUN_NOWAIT)
      break;
  }

  /* The if statement lets gcc compile it to a conditional store. Avoids
   * dirtying a cache line.
   */
  if (loop->stop_flag != 0)
    loop->stop_flag = 0;

  return r;
}

timers phase

执行setTimeout() 和 setInterval()中到期的callback

void uv__run_timers(uv_loop_t* loop) {
  struct heap_node* heap_node;
  uv_timer_t* handle;

  for (;;) {
    heap_node = heap_min(timer_heap(loop));
    if (heap_node == NULL)
      break;
		// 取出堆中最快要被执行的timer
    // #define container_of(ptr, type, member) 
  	// ((type *) ((char *) (ptr) - offsetof(type, member)))
    // 没看懂 handle是怎么生成的
    handle = container_of(heap_node, uv_timer_t, heap_node);
    if (handle->timeout > loop->time)//执行时间大于eventloop循环一次时间，退出phase下次再说
      break;

    uv_timer_stop(handle);// remove handle
    uv_timer_again(handle);// 多次重复的timer再塞进去
    handle->timer_cb(handle);// invoke callback
  }
}

I/O callbacks

上一轮循环中有少数的I/Ocallback会被延迟到这一轮的这一阶段执行

//deps/uv/src/unix/core.c
static int uv__run_pending(uv_loop_t* loop) {
  QUEUE* q;
  QUEUE pq;
  uv__io_t* w;

  if (QUEUE_EMPTY(&loop->pending_queue))//isEmpty
    return 0;

  QUEUE_MOVE(&loop->pending_queue, &pq);//move

  while (!QUEUE_EMPTY(&pq)) {
    q = QUEUE_HEAD(&pq);//find
    QUEUE_REMOVE(q);//pop
    QUEUE_INIT(q);
    w = QUEUE_DATA(q, uv__io_t, pending_queue);
    w->cb(loop, w, POLLOUT);//unitl queue empty
  }

  return 1;
}

Idle and prepare phase

/* loop */

void uv__run_idle(uv_loop_t* loop);

void uv__run_check(uv_loop_t* loop);

void uv__run_prepare(uv_loop_t* loop);

void uv__run_##name(uv_loop_t* loop) {
  uv_##name##_t* h;
  QUEUE queue;
  QUEUE* q;
  QUEUE_MOVE(&loop->name##_handles, &queue);//QUEUE_MOVE
  while (!QUEUE_EMPTY(&queue)) {//util empty
    q = QUEUE_HEAD(&queue);//pop
    h = QUEUE_DATA(q, uv_##name##_t, queue);//element->handle
    QUEUE_REMOVE(q);//remove
    QUEUE_INSERT_TAIL(&loop->name##_handles, q);//insert tail
    h->name##_cb(h);//callback
  }
}

!!!poll phase!!!

最为重要的阶段，执行I/O callback，在适当的条件下会阻塞在这个阶段

可见poll阶段的任务就是阻塞等待监听的事件来临，然后执行对应的callback，其中阻塞是带有超时时间的，以下几种情况都会使得超时时间为0

uv_run处于UV_RUN_NOWAIT模式下

uv_stop()被调用

没有活跃的handles和request

有活跃的idle handles

有等待关闭的handles

如果上述都不符合，则超时时间为距离现在最近的timer；如果没有timer则poll阶段会一直阻塞下去

个人理解nodejs的服务，大部分时间会被阻塞在这个阶段，而不去执行closing

// 不行了，看不懂了
void uv__io_poll(uv_loop_t* loop, int timeout) {
  struct pollfd events[1024];
  struct pollfd pqry;
  struct pollfd* pe;
  struct poll_ctl pc;
  QUEUE* q;
  uv__io_t* w;
  uint64_t base;
  uint64_t diff;
  int have_signals;
  int nevents;
  int count;
  int nfds;
  int i;
  int rc;
  int add_failed;

  if (loop->nfds == 0) {
    assert(QUEUE_EMPTY(&loop->watcher_queue));
    return;
  }

  while (!QUEUE_EMPTY(&loop->watcher_queue)) {//until watcher queue empty
    q = QUEUE_HEAD(&loop->watcher_queue);
    QUEUE_REMOVE(q);
    QUEUE_INIT(q);

    w = QUEUE_DATA(q, uv__io_t, watcher_queue);
    assert(w->pevents != 0);
    assert(w->fd >= 0);
    assert(w->fd < (int) loop->nwatchers);

    pc.events = w->pevents;
    pc.fd = w->fd;

    add_failed = 0;
    if (w->events == 0) {
      pc.cmd = PS_ADD;
      if (pollset_ctl(loop->backend_fd, &pc, 1)) {
        if (errno != EINVAL) {
          assert(0 && "Failed to add file descriptor (pc.fd) to pollset");
          abort();
        }
        /* Check if the fd is already in the pollset */
        pqry.fd = pc.fd;
        rc = pollset_query(loop->backend_fd, &pqry);
        switch (rc) {
        case -1:
          assert(0 && "Failed to query pollset for file descriptor");
          abort();
        case 0:
          assert(0 && "Pollset does not contain file descriptor");
          abort();
        }
        /* If we got here then the pollset already contained the file descriptor even though
         * we didn't think it should. This probably shouldn't happen, but we can continue. */
        add_failed = 1;
      }
    }
    if (w->events != 0 || add_failed) {
      /* Modify, potentially removing events -- need to delete then add.
       * Could maybe mod if we knew for sure no events are removed, but
       * content of w->events is handled above as not reliable (falls back)
       * so may require a pollset_query() which would have to be pretty cheap
       * compared to a PS_DELETE to be worth optimizing. Alternatively, could
       * lazily remove events, squelching them in the mean time. */
      pc.cmd = PS_DELETE;
      if (pollset_ctl(loop->backend_fd, &pc, 1)) {
        assert(0 && "Failed to delete file descriptor (pc.fd) from pollset");
        abort();
      }
      pc.cmd = PS_ADD;
      if (pollset_ctl(loop->backend_fd, &pc, 1)) {
        assert(0 && "Failed to add file descriptor (pc.fd) to pollset");
        abort();
      }
    }

    w->events = w->pevents;
  }

  assert(timeout >= -1);
  base = loop->time;
  count = 48; /* Benchmarks suggest this gives the best throughput. */

  for (;;) {
    nfds = pollset_poll(loop->backend_fd,
                        events,
                        ARRAY_SIZE(events),
                        timeout);

    /* Update loop->time unconditionally. It's tempting to skip the update when
     * timeout == 0 (i.e. non-blocking poll) but there is no guarantee that the
     * operating system didn't reschedule our process while in the syscall.
     */
    SAVE_ERRNO(uv__update_time(loop));

    if (nfds == 0) {
      assert(timeout != -1);
      return;
    }

    if (nfds == -1) {
      if (errno != EINTR) {
        abort();
      }

      if (timeout == -1)
        continue;

      if (timeout == 0)
        return;

      /* Interrupted by a signal. Update timeout and poll again. */
      goto update_timeout;
    }

    have_signals = 0;
    nevents = 0;

    assert(loop->watchers != NULL);
    loop->watchers[loop->nwatchers] = (void*) events;
    loop->watchers[loop->nwatchers + 1] = (void*) (uintptr_t) nfds;

    for (i = 0; i < nfds; i++) {
      pe = events + i;
      pc.cmd = PS_DELETE;
      pc.fd = pe->fd;

      /* Skip invalidated events, see uv__platform_invalidate_fd */
      if (pc.fd == -1)
        continue;

      assert(pc.fd >= 0);
      assert((unsigned) pc.fd < loop->nwatchers);

      w = loop->watchers[pc.fd];

      if (w == NULL) {
        /* File descriptor that we've stopped watching, disarm it.
         *
         * Ignore all errors because we may be racing with another thread
         * when the file descriptor is closed.
         */
        pollset_ctl(loop->backend_fd, &pc, 1);
        continue;
      }

      /* Run signal watchers last.  This also affects child process watchers
       * because those are implemented in terms of signal watchers.
       */
      if (w == &loop->signal_io_watcher)
        have_signals = 1;
      else
        w->cb(loop, w, pe->revents);

      nevents++;
    }

    if (have_signals != 0)
      loop->signal_io_watcher.cb(loop, &loop->signal_io_watcher, POLLIN);

    loop->watchers[loop->nwatchers] = NULL;
    loop->watchers[loop->nwatchers + 1] = NULL;

    if (have_signals != 0)
      return;  /* Event loop should cycle now so don't poll again. */

    if (nevents != 0) {
      if (nfds == ARRAY_SIZE(events) && --count != 0) {
        /* Poll for more events but don't block this time. */
        timeout = 0;
        continue;
      }
      return;
    }

    if (timeout == 0)
      return;

    if (timeout == -1)
      continue;

update_timeout:
    assert(timeout > 0);

    diff = loop->time - base;
    if (diff >= (uint64_t) timeout)
      return;

    timeout -= diff;
  }
}

check phase

见idle prepare

close

关闭handle

static void uv__run_closing_handles(uv_loop_t* loop) {
  uv_handle_t* p;
  uv_handle_t* q;

  p = loop->closing_handles;
  loop->closing_handles = NULL;

  while (p) {
    q = p->next_closing;
    uv__finish_close(p);
    p = q;
  }
}

where is process.nextTick

//lib/internal/process/task_queues.js
// `nextTick()` will not enqueue any callback when the process is about to
// exit since the callback would not have a chance to be executed.
// 意思就是nextTick在进程快要结束时不会排队callback,因为没有机会执行
// 你们看引用的文档吧，我看不下去了😭
// 主要的思路是JS执行process.nexTick()，然后将callback交给c++执行
function nextTick(callback) {
  if (typeof callback !== 'function')
    throw new ERR_INVALID_CALLBACK(callback);

  if (process._exiting)
    return;

  let args;
  switch (arguments.length) {
    case 1: break;
    case 2: args = [arguments[1]]; break;
    case 3: args = [arguments[1], arguments[2]]; break;
    case 4: args = [arguments[1], arguments[2], arguments[3]]; break;
    default:
      args = new Array(arguments.length - 1);
      for (let i = 1; i < arguments.length; i++)
        args[i - 1] = arguments[i];
  }

  if (queue.isEmpty())
    setHasTickScheduled(true);
  const asyncId = newAsyncId();
  const triggerAsyncId = getDefaultTriggerAsyncId();
  const tickObject = {
    [async_id_symbol]: asyncId,
    [trigger_async_id_symbol]: triggerAsyncId,
    callback,
    args
  };
  if (initHooksExist())
    emitInit(asyncId, 'TickObject', triggerAsyncId, tickObject);
  queue.push(tickObject);//封装callback push
  //进入c
}

question

1.setTimeout vs setImmediate

phase执行顺序
expire设置0是不是立刻执行

setTimeout(() => {
  console.log('setTimeout')
}, 0)

setImmediate(() => {
  console.log('setImmediate')
})

setTimeout/setInterval 的第二个参数取值范围是：[1, 2^31 - 1]，如果超过这个范围则会初始化为 1，即 setTimeout(fn, 0) === setTimeout(fn, 1)。
setTimeout 的回调函数在 timer 阶段执行，setImmediate 的回调函数在 check 阶段执行，event loop 的开始会先检查 timer 阶段，但是在开始之前到 timer 阶段会消耗一定时间，所以就会出现两种情况：
- timer 前的准备时间超过 1ms，满足 loop->time >= 1，则执行 timer 阶段（setTimeout）的回调函数
- timer 前的准备时间小于 1ms，则先执行 check 阶段（setImmediate）的回调函数，下一次 event loop 执行 timer 阶段（setTimeout）的回调函数

在举例：

setTimeout(() => {
  console.log('setTimeout')
}, 0)

setImmediate(() => {
  console.log('setImmediate')
})

const start = Date.now()
while (Date.now() - start < 10);//准备时间超过1ms，则直接执行timer

2.setTimeout vs setImmediate 2

const fs = require('fs')

fs.readFile(__filename, () => {
  setTimeout(() => {
    console.log('setTimeout')
  }, 0)

  setImmediate(() => {
    console.log('setImmediate')
  })
})
//setImmediate
//setTimeout

在引用一下官方对于check phase的介绍

This phase allows a person to execute callbacks immediately after the poll phase has completed. If the poll phase becomes idle and scripts have been queued with setImmediate(), the event loop may continue to the check phase rather than waiting.

setImmediate() is actually a special timer that runs in a separate phase of the event loop. It uses a libuv API that schedules callbacks to execute after the poll phase has completed.

Generally, as the code is executed, the event loop will eventually hit the poll phase where it will wait for an incoming connection, request, etc. However, if a callback has been scheduled with setImmediate() and the poll phase becomes idle, it will end and continue to the check phase rather than waiting for poll events.

fs.readFile 的回调函数执行完后：

注册 setTimeout 的回调函数到 timer 阶段
注册 setImmediate 的回调函数到 check 阶段
event loop 从 pool 阶段出来继续往下一个阶段执行，恰好是 check 阶段，所以 setImmediate 的回调函数先执行
本次 event loop 结束后，进入下一次 event loop，执行 setTimeout 的回调函数

所以，在 I/O Callbacks 中注册的 setTimeout 和 setImmediate，永远都是 setImmediate 先执行。

3.process.nextTick()

setInterval(() => {
  console.log('setInterval')
}, 100)

process.nextTick(function tick () {
  process.nextTick(tick)
})
//note
setImmediate(function immediate () {
  console.log('111');//会直接打印出很多次111
  setImmediate(immediate)
})

运行结果：setInterval 永远不会打印出来。

//这个在node官方文档也有相关的描述

//我在这里也进行了笔记记录

//允许用户处理errors，清理不需要的资源，事件循环前尝试重新连接

//有时有必要在eventloop继续之前，在call stack unwound之后，让callback执行

解释：process.nextTick 会无限循环，将 event loop 阻塞在 microtask 阶段，导致 event loop 上其他 macrotask 阶段的回调函数没有机会执行。//这段解释是前端的，后端是没有microtask的实际队列的

解决方法通常是用 setImmediate 替代 process.nextTick，如下：

setInterval(() => {
  console.log('setInterval')
}, 100)

setImmediate(function immediate () {
  setImmediate(immediate)
})

运行结果：每 100ms 打印一次 setInterval。

解释：process.nextTick 内执行 process.nextTick 仍然将 tick 函数注册到当前 microtask 的尾部，所以导致 microtask 永远执行不完； setImmediate 内执行 setImmediate 会将 immediate 函数注册到下一次 event loop 的 check 阶段，而不是当前正在执行的 check 阶段，所以给了 event loop 上其他 macrotask 执行的机会。

再看个例子：

setImmediate(() => {
  console.log('setImmediate1')
  setImmediate(() => {
    console.log('setImmediate2')
  })
  process.nextTick(() => {
    console.log('nextTick')
  })
})

setImmediate(() => {
  console.log('setImmediate3')
})

运行结果：

setImmediate1
setImmediate3
nextTick
setImmediate2

注意：并不是说 setImmediate 可以完全替代 process.nextTick，process.nextTick 在特定场景下还是无法被替代的，比如我们就想将一些操作放到最近的 microtask 里执行。

4.promise

const promise = Promise.resolve()
  .then(() => {
    return promise
  })
promise.catch(console.error)

运行结果：

TypeError: Chaining cycle detected for promise #<Promise>
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
    at Function.Module.runMain (module.js:667:11)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:607:3

解释：promise.then 类似于 process.nextTick，都会将回调函数注册到 microtask 阶段。上面代码会导致死循环，类似前面提到的：

1
2
3

process.nextTick(function tick () {
  process.nextTick(tick)
})

再看个例子：

const promise = Promise.resolve()

promise.then(() => {
  console.log('promise')
})

process.nextTick(() => {
  console.log('nextTick')
})

运行结果：

1 2	nextTick promise

解释：promise.then 虽然和 process.nextTick 一样，都将回调函数注册到 microtask，但优先级不一样。process.nextTick 的 microtask queue 总是优先于 promise 的 microtask queue 执行。

5.promise执行顺序

setTimeout(() => {
  console.log(1)
}, 0)

new Promise((resolve, reject) => {
  console.log(2)
  for (let i = 0; i < 10000; i++) {
    i === 9999 && resolve()
  }
  console.log(3)
}).then(() => {
  console.log(4)
})
console.log(5)

运行结果：

解释：Promise 构造函数是同步执行的，所以先打印 2、3，然后打印 5，接下来 event loop 进入执行 microtask 阶段，执行 promise.then 的回调函数打印出 4，然后执行下一个 macrotask，恰好是 timer 阶段的 setTimeout 的回调函数，打印出 1。

6.综合

setImmediate(() => {
  console.log(1)
  setTimeout(() => {
    console.log(2)
  }, 100)
  setImmediate(() => {
    console.log(3)
  })
  process.nextTick(() => {
    console.log(4)
  })
})
process.nextTick(() => {
  console.log(5)
  setTimeout(() => {
    console.log(6)
  }, 100)
  setImmediate(() => {
    console.log(7)
  })
  process.nextTick(() => {
    console.log(8)
  })
})
console.log(9)

运行结果：

process.nextTick、setTimeout 和 setImmediate 的组合，请读者自己推理吧。

other source code

setTimeout()

//lib/timers/promises.js
//setTimeout(function(){},expire)
function setTimeout(after, value, options = {}) {
  const args = value !== undefined ? [value] : value;
  if (options == null || typeof options !== 'object') {
    return PromiseReject(
      new ERR_INVALID_ARG_TYPE(
        'options',
        'Object',
        options));
  }
  const { signal, ref = true } = options;
  if (signal !== undefined &&
      (signal === null ||
       typeof signal !== 'object' ||
       !('aborted' in signal))) {
    return PromiseReject(
      new ERR_INVALID_ARG_TYPE(
        'options.signal',
        'AbortSignal',
        signal));
  }
  if (typeof ref !== 'boolean') {
    return PromiseReject(
      new ERR_INVALID_ARG_TYPE(
        'options.ref',
        'boolean',
        ref));
  }
  // TODO(@jasnell): If a decision is made that this cannot be backported
  // to 12.x, then this can be converted to use optional chaining to
  // simplify the check.
  if (signal && signal.aborted)
    return PromiseReject(lazyDOMException('AbortError'));
  return new Promise((resolve, reject) => {
    const timeout = new Timeout(resolve, after, args, false, true);
    if (!ref) timeout.unref();
    insert(timeout, timeout._idleTimeout);
    if (signal) {
      signal.addEventListener('abort', () => {
        if (!timeout._destroyed) {
          // eslint-disable-next-line no-undef
          clearTimeout(timeout);
          reject(lazyDOMException('AbortError'));
        }
      }, { once: true });
    }
  });
}

算法图解

Posted on 2020-06-08 Edited on 2024-05-30 In Books

1. 算法简介

1.1 二分

Why: 复杂度O(n)—>O(logn)

使用限制：有序数组

1.2 大O表示

指出算法运行时间的增速，算法需要做的就是把O(n^2)优化到O(n)等

2. 选择排序

2.1 数组链表

数组：连续物理空间，可随机访问，增删数据复杂度高

链表：分散无力空间，不可随机访问（只能顺序），增删数据复杂度低

	数组	链表
读改	O(1)	O(n)
增删	O(n)	O(1)

根据互相特性，选择合适的方式，如频繁增删用链表，反之用数组

2.2 选择排序

复杂度：O(n^2)

遍历n 个元素选择最小/大的，遍历n-1个元素选择最小/大的

3. 递归

类比：套娃 :call_me_hand:

性能和易读不可兼得

避免死循环！

尾递归可以解决部分性能问题

递归调用栈是性能降低的原因，遵循FIFO

4. 快排

核心：分而治之divide and conquer，快排只是其中的一个应用

思想：递归的一种应用

快排（递归）是一种函数式编程

快排通过基准值（可以选第一个元素）进行分而治之

5. 散列表

实现方式：数组，非链表，检索值key类似数组的下表，可直接访问value

应用：DNS，阻止重复数据(类set集)，作缓存（服务器端）

复杂度	散列平均	散列最糟	数组	链表
查找	1	n	1	n
插入删除	1	n	n	1

装填因子（0.4）=散列元素（4）/位置总数（10）

避免冲突：1.良好的散列函数（均匀分布） 2.较低的装填因子（<0.7）

将满时候：1.申请两倍于原来的新空间 2.hash所有元素到新空间

冲突解决：

开放地址（最简单就是冲突顺延下一位，直到为空）
拉链发（指在某个位子上再拉一条链表，非👖拉链）

6.BFS

广度优先搜索breadth first search，解决无加权最短路径问题之一

应用：国际跳棋，拼写检查，人际关系网络

7. Dijkstra

正加权有向无环图的解决算法

最短时间内到达的节点
更新该节点临接节点的开销
重复
计算最终路径

解决环：

负加权：bellman ford algorithm

8. Greedy

每步最优–>全局最优，得到近似正确的结果

9.DP

列出所有可能

10. K最邻近算法

11.next

树解决了二分查找中，插入删除O(n)降低到O(log n)，但是降低了随机访问能力

树包括：二叉树，平衡二叉树，B树 B+树，，红黑树

反向索引：散列表，用于创建搜索引擎—>应用：傅里叶变换

并行算法，单机并行or分布式，应用：mapreduce，map->映射，reduce->归并

布隆过滤器：庞大的散列表（如谷歌的上亿条），通常使用redis实现，是一种概率型数据结构（偶尔出错），使用理由，存储空间少

hyperLogLog：类似布隆，是个日志

SHA算法

散列的一种应用
判断两个（超大）文件是否相同（散列值相同）
SHA（用户输入密码）？== 数据库存储的SHA值，且拖库后无法还原密码
SHA是一系列算法的统称，包括SHA-0 ,SHA-1 SHA-2 SHA-3 bcrypt etc
SHA全局敏感（改动局部，整体全变），SIMhash局部敏感（局部改变，散列值局部改变），后者用于判断网页是否已经搜集，作业是否抄袭，相似度查询
diffie-hellman密钥交换
- 双方无需知道加密算法，破解难度大
- 公钥与私钥，client获取公钥后，1.使用公钥加密 2.服务器端使用私钥解密

线性规划：simplex算法

What

Promise 对象有以下两个特点:

Why

How

what

why

how

what

why

how

thanks

what

why

how

1.docker run

2.Usage

eventLoop

eventLoop

timers phase

I/O callbacks

Idle and prepare phase

!!!poll phase!!!

check phase

close

where is process.nextTick

question

1.setTimeout vs setImmediate

2.setTimeout vs setImmediate 2

3.process.nextTick()

4.promise

5.promise执行顺序

6.综合

other source code

setTimeout()

1. 算法简介

1.1 二分

1.2 大O表示

2. 选择排序

2.1 数组 链表

2.2 选择排序

3. 递归

4. 快排

5. 散列表

6.BFS

7. Dijkstra

8. Greedy

9.DP

10. K最邻近算法

11.next

2.1 数组链表