0%

This is my first blog.

My name is Javen.

My blog will write something about my life, my work.

ok then hope you enjoy my next blog.

Linux


码云

1. 添加公钥

生成/添加 SSH 公钥

1
ssh-keygen -t rsa -C "[email protected]"

SSH 公钥设置

1
ssh -T [email protected]

2.创建仓库

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 全局配置用户名和邮箱
git config --global user.name "xxx"
git config --global user.email "[email protected]"

# 没有现成的项目
mkdir xxxx
cd xxxx
git init
touch README.md
git add README.md
git commit -m "first commit"
git remote add origin https://gitee.com/xxx/xxxx
git push -u origin master

# 有现成的项目
cd xxxx
git remote add origin https://gitee.com/xxx/xxxx
git push -u origin master

Git 基本操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# 检查状态,每次操作最好都要查看状态
git status

# 本地有文件没有提交的话,需要将未提交文件添加到到暂存区(红色为未添加到暂存区的,绿色表示正常)
git add .

# 本地提交
git commit -m "xxxx"

# 关联本地仓库和码云仓库
cd xxxx
git remote add origin https://gitee.com/xxx/xxxx
git push -u origin master
# 输入码云账号密码

# 创建分支
git checkout -b 分支名称

# 查看所有分支 前面带*位当前分支
git branch

# 提交本地代码到码云
git push

# 合并分支到master分支
# 1、切换到master分支
git checkout master
git branch
# 2、本地合并分支到master分支
git merge xxxx # xxxx代表分支名称
# 3、提交本地master分支代码到码云
git push
# 4、本地分支推送到码云
git checkout xxxx # xxxx代表分支名称
git branch
git push -u origin xxxx # 第一次推送分支代码到码云,xxxx代表分支名称

# 代码测试没问题,需要本地合并子分支到主分支,将主分支上传到码云,最后将分支也上传到码云

git clone -b xxx url # 克隆xxx分支代码


Git 忽略不想提交的文件

1
2
3
4
5
6
7
8
9
# 1.使用.gitignore 要求需要忽略的文件是未追踪状态,也就是说没有被git add 过
# 2.如果已经add 过可以使用以下命令取消追踪
# 会删除文件的追踪状态,不会删除本地文件,git push后是会删除远程仓库内的文件的
git rm --cached file/path
#3.需要忽略的文件是仓库内必须的,但需要忽略本地改动(也就是说远程仓库中不能删除),如一些配置文件
git update-index --assume-unchanged file/path
#这个命令可以忽略已经是追踪状态的文件,但是无法将这个忽略状态提交至远程仓库,只对当前本地工程有效
#取消忽略:
git update-index --no-assume-unchanged file/path

PowerShell 解决中文乱码

1、在 PowerShell 中输入以下命令

1
2
3
4
5
git config --global core.quotepath false
git config --global gui.encoding utf-8
git config --global i18n.commit.encoding utf-8
git config --global i18n.logoutputencoding utf-8
$env:LESSCHARSET='utf-8'

2、在系统环境变量中添加变量 LESSCHARSET 为 utf-8.

git pull 时解决分支分叉(branch diverged)问题

git pull 时出现分支冲突(branch diverged)

1
2
3
4
$ git status
# On branch feature/worker-interface
# Your branch and 'origin/br_dev' have diverged,
# and have 1 and 4 different commit(s) each, respectively.

解决办法:
1、查看哪个 commit 冲突
git cherry origin/br_dev

2、回退到冲突的前一个 commit
git reset –hard

3、更新代码
git pull

Git 撤销&回滚操作(git reset 和 get revert)

Linux


1
2
3
4
5
6
7
# 压缩
tar –cvf jpg.tar *.jpg # 将目录里所有jpg文件打包成tar.jpg
tar –czf jpg.tar.gz *.jpg # 将目录里所有jpg文件打包成jpg.tar后,并且将其用gzip压缩,生成一个gzip压缩过的包,命名为jpg.tar.gz

# 解压
tar –xvf file.tar # 解压 tar包
tar -xzvf file.tar.gz # 解压tar.gz
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# JDK8安装

wget https://code.aliyun.com/kar/oracle-jdk/raw/3c932f02aa11e79dc39e4a68f5b0483ec1d32abe/jdk-8u251-linux-x64.tar.gz

tar zxvf jdk-8u251-linux-x64.tar.gz

vim /etc/profile

JAVA_HOME=/server/jdk1.8.0_251
JRE_HOME=$JAVA_HOME/jre
PATH=$PATH:$JAVA_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME
export JRE_HOME
export PATH
export CLASSPATH

source /etc/profile

ln -s /server/jdk1.8.0_251/bin/java /usr/bin/java
1
2
3
nohup java -jar blog-backend-1.0-SNAPSHOT.jar --server.port=8888 &

nohup java -jar blog-backend-1.0-SNAPSHOT.jar --spring.config.location=/server/application-server-sqlite.yml -Xms64m -Xmx128m -Xss256k -XX:ParallelGCThreads=2 -Djava.compiler=NONE &
1
2
3
4
5
6
7
# 查看运行状态
netstat -lntup | grep nginx
# 查看日志
tail -n 50 nohup.out
tail -f nohup.out
# 查看内存占用前10
ps -aux | sort -k4nr | head -10
1
2
3
4
5
6
7
8
9
10
11
12
13
server {
listen 80;
server_name nginx.test.com;

location / {
proxy_pass http://localhost:8080;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
#root html;
#index index.html index.htm;
}
}
1
2
3
4
5
git clone https://github.com/letsencrypt/letsencrypt

cd letsencrypt/

./letsencrypt-auto certonly --standalone --email [email protected] -d jwhuang.cn -d jwhuang.cn

Linux


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# docker容器开机自启动
--restart=always

docker run -di --name=registry --restart=always -p 5000:5000 registry

# rabbitmq安装
sudo docker run -di --name=rabbitmq --restart=always -p 5671:5671 -p 5672:5672 -p 4369:4369 -p 15671:15671 -p 15672:15672 -p 25672:25672 rabbitmq:management

sudo docker run -di -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 --name influxsrv tutum/influxdb

sudo docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --publish=7001:8080 --detach=true --link influxsrv:influxsrv --name=cadvisor google/cadvisor -storage_driver=influxdb -storage_driver_db=tensquare-db -storage_driver_host=influxsrv:8086

docker run -d -p 3001:3000 -e INFLUXDB_HOST=influxsrv -e INFLUXDB_PORT=8086 -e INFLUXDB_NAME=cadvisor -e INFLUXDB_USER=javen -e INFLUXDB_PASS=123456 --link influxsrv:influxsrv --name=grafana grafana/grafana

docker run -di --restart=always -p 10022:22 -p 10080:3000 --name=gogs --net=backend -v /opt/docker/gogs/:/data gogs/gogs

docker run -di --restart=always -p 9001:8080 \
-v /opt/docker/jenkins/workspace:/root/.jenkins/workspace \
-v /opt/docker/jenkins/run/docker.sock:/var/run/docker.sock \
-v /usr/bin/git:/usr/bin/git \
-v /usr/local/jdk1.8:/usr/local/jdk1.8 \
-v /usr/local/maven:/usr/local/maven3 --name jenkins jenkins:latest

mkdir -p /opt/docker/nginx/www /home/nginx/logs /home/nginx/conf
docker run --rm -d -p 8081:80 --name nginx \
-v /opt/docker/nginx/www:/usr/share/nginx/html \
-v /opt/docker/nginx:/etc/nginx \
-v /opt/docker/nginx/logs:/var/log/nginx \
nginx

docker run -d -v /opt/docker/oracle_data:/data/oracle_data --name oracle -p 49159:8080 -p 49160:22 -p 49161:1521 -e ORACLE_ALLOW_REMOTE=true deepdiver/docker-oracle-xe-11g:latest
1
docker exec -it 243c32535da7 /bin/bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
#依赖镜像名称和ID
FROM centos:7
#指定镜像创建者信息
MAINTAINER JAVEN
#切换工作目录
WORKDIR /usr
RUN mkdir /usr/local/java
#ADD 是相对路径jar,把java添加到容器中
ADD jdk-8u191-linux-x64.tar.gz /usr/local/java/
#配置java环境变量
ENV JAVA_HOME /usr/local/java/jdk1.8.0_191
ENV JRE_HOME $JAVA_HOME/jre
ENV CLASSPATH $JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:$CLASSPATH
ENV PATH $JAVA_HOME/bin:$PATH
1
2
3
4
5
6
7
8
9
10
11
FROM jdk1.8
# 作者
MAINTAINER javen <[email protected]>
# VOLUME 指定了临时文件目录为/tmp。
# 其效果是在主机 /var/lib/docker 目录下创建了一个临时文件,并链接到容器的/tmp
VOLUME /tmp
# 将jar包添加到容器中并更名为app.jar
ADD demo-0.0.1-SNAPSHOT.jar app.jar
# 运行jar包
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

  • 添加 pom
1
2
3
4
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
  • 配置

    spring.rabbitmq.host=10.110.3.62
    spring.rabbitmq.port=5672
    spring.rabbitmq.username=guest
    spring.rabbitmq.password=guest
    spring.rabbitmq.virtual-host=/
    spring.rabbitmq.listener.simple.concurrency= 10
    spring.rabbitmq.listener.simple.max-concurrency= 10
    spring.rabbitmq.listener.simple.prefetch= 1
    spring.rabbitmq.listener.simple.auto-startup=true
    spring.rabbitmq.listener.simple.default-requeue-rejected= true
    spring.rabbitmq.template.retry.enabled=true
    spring.rabbitmq.template.retry.initial-interval=1000
    spring.rabbitmq.template.retry.max-attempts=3
    spring.rabbitmq.template.retry.max-interval=10000
    spring.rabbitmq.template.retry.multiplier=1.0

首先在父工程定义好 SpringCloud 版本

1
2
3
4
5
6
7
8
9
10
11
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Finchley.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

服务发现——Netflix Eureka

创建 Eureka 模块
pom 文件添加
1
2
3
4
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
application.yml 添加配置
1
2
3
4
5
6
7
8
server:
port: 6868
eureka:
client:
register-with-eureka: false
fetch-registry: false
service-url:
defaultZone: http://127.0.0.1:${server.port}/eureka/
启动类上添加注解
1
@EnableEurekaServer
其他模块注册到 Eureka 上面
pom 文件添加
1
2
3
4
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
application.yml 添加配置
1
2
3
4
5
6
7
8
server:
port: 6868
eureka:
client:
service-url:
defaultZone: http://127.0.0.1:6868/eureka/
instance:
prefer-ip-address: true
启动类上添加注解
1
@EnableEurekaClient

服务调用——Netflix Feign

在需要的调用其他模块的模块中修改

pom 文件添加
1
2
3
4
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
application.yml 添加配置
1
2
3
4
5
6
7
8
9
10
11
server:
port: 9001
spring:
application:
name: tensquare-base
eureka:
client:
service-url:
defaultZone: http://127.0.0.1:6868/eureka/
instance:
prefer-ip-address: true
启动类上添加注解
1
2
@EnableDiscoveryClient
@EnableFeignClients

创建 client 包,里面创建接口

1
2
3
4
5
6
7
@FeignClient(value = "TENSQUARE-BASE", fallback = BaseClientImpl.class)
public interface BaseClient {

@RequestMapping(value = "/label/{labelId}", method = RequestMethod.GET)
public Result findById(@PathVariable("labelId") String labelId);

}

熔断器——Netflix Hystrix

application.yml 添加配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
server:
port: 9001
spring:
application:
name: tensquare-base
eureka:
client:
service-url:
defaultZone: http://127.0.0.1:6868/eureka/
instance:
prefer-ip-address: true
feign:
hystrix:
enabled: true
创建 client 接口实现类
1
2
3
4
5
6
7
@Component
public class BaseClientImpl implements BaseClient {
@Override
public Result findById(String labelId) {
return new Result(false, StatusCode.ERROR, "熔断器");
}
}
修改之前写的接口上的注解,加入 fallback 参数为实现类
1
2
3
4
5
6
7
@FeignClient(value = "TENSQUARE-BASE", fallback = BaseClientImpl.class)
public interface BaseClient {

@RequestMapping(value = "/label/{labelId}", method = RequestMethod.GET)
public Result findById(@PathVariable("labelId") String labelId);

}

服务网关——Netflix Zuul

创建网关模块
pom 文件添加
1
2
3
4
5
6
7
8
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>
application.yml 添加配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
server:
port: 9011
spring:
application:
name: tensquare-manager
eureka:
client:
service-url:
defaultZone: http://127.0.0.1:6868/eureka/
instance:
prefer-ip-address: true
zuul:
routes:
TENSQUARE-BASE:
path: /base/**
serviceId: TENSQUARE-BASE
TENSQUARE-USER:
path: /user/**
serviceId: TENSQUARE-USER
TENSQUARE-QA:
path: /qa/**
serviceId: TENSQUARE-QA
启动类上添加注解
1
@EnableZuulProxy
Zuul 过滤器
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
@Component
public class WebFilter extends ZuulFilter {
@Override
public String filterType() {
return "pre";//配置前置过滤器
}

@Override
public int filterOrder() {
return 0;//优先级为0,数字越小,优先级越高
}

@Override
public boolean shouldFilter() {
return true;//是否启用
}

@Override
public Object run() throws ZuulException {
// header经过网关会丢失,所以需要header的情况要接受header信息转发给微服务

//直接转发
RequestContext currentContext = RequestContext.getCurrentContext();
HttpServletRequest request = currentContext.getRequest();
String authorization = request.getHeader("Authorization");
if (authorization != null && !"".equals(authorization)) {
currentContext.addZuulRequestHeader("Authorization", authorization);
}
return null;

// 需要鉴权的情况
RequestContext currentContext = RequestContext.getCurrentContext();
HttpServletRequest request = currentContext.getRequest();

if (request.getMethod().equals("OPTIONS")) {
return null;
}

if (request.getRequestURL().indexOf("login") > 0) {
return null;
}


String authorization = request.getHeader("Authorization");
if (authorization != null && !"".equals(authorization)) {
if (authorization.startsWith("Bearer")) {
String token = authorization.substring(7);
System.out.println(token);
try {
Claims claims = jwtUtil.parseJWT(token);
String roles = (String) claims.get("roles");
System.out.println(roles);
if (roles.equals("admin")) {
currentContext.addZuulRequestHeader("Authorization", authorization);
return null;
}
} catch (Exception e) {
currentContext.setSendZuulResponse(false);
}
}
}
currentContext.setSendZuulResponse(false);//写了这一行,不会转发到微服务
currentContext.setResponseStatusCode(403);
currentContext.setResponseBody("权限不足");
currentContext.getResponse().setContentType("text/html;charset=utf-8");
return null;
}
}

分布式配置——Spring Cloud Config

将配置文件提交到码云 格式为{application}-{profile}.yml,例如 base-dev.yml
开启 RebbitMQ
创建配置文件微服务
pom 文件添加
1
2
3
4
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
application.yml 添加配置
1
2
3
4
5
6
7
8
9
10
server:
port: 12000
spring:
application:
name: tensquare-config
cloud:
config:
server:
git:
uri: https://gitee.com/jwhuang10/tensquare_dev.git # 码云地址
启动类上添加注解
1
@EnableConfigServer
配置客户端
pom 文件添加
1
2
3
4
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
添加 bootstrap.yml ,删除 application.yml
1
2
3
4
5
6
7
spring:
cloud:
config:
name: base
profile: dev
label: master
uri: http://127.0.0.1:12000 # 服务端地址

消息总线 —— Spring Cloud Bus

修改配置文件微服务
pom 文件添加
1
2
3
4
5
6
7
8
9
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-bus</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-rabbit</artifactId>
</dependency>

修改 application.yml

1
2
3
4
5
6
7
8
9
10
  rabbitmq:
host: 192.168.0.10
management:
endpoints:
web:
exposure:
include: "*"
endpoint:
bus-refresh:
enabled: true
配置客户端
pom 文件添加
1
2
3
4
5
6
7
8
9
10
11
12
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-bus</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-rabbit</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
修改码云上面对应的 application.yml,添加 rabbitmq 配置
1
2
3
4
5
6
7
server:
port: 9001
spring:
application:
name: tensquare-base
rabbitmq:
host: 192.168.0.10

每次修改码云上的配置文件,发送 POST 请求到http://127.0.0.1:12000/actuator/bus-refresh,即可修改微服务配置

修改 application.yml 的自定义配置项

需要在 controller 上添加以下注解

1
@RefreshScope

nginx

1.下载 nginx-x.xx.x.tar.gz

2.解压

1
tar xvf nginx-x.xx.x.tar.gz

3.编译

1
2
3
4
5
6
# 进入目录
cd nginx-x.xx.x
# 配置
./configure --prefix=/opt/nginx --sbin-path=/usr/bin/nginx # 安装目录 /opt/nginx
# 编译安装
make && make install

4.查看 nginx 状态

1
ps -ef | grep nginx

5.关闭防火墙

1
2
3
4
# 开机不启动
chkconfig iptables off
# 临时关闭
service iptables stop

6.nginx 启动

1
2
3
4
5
6
# 启动
nginx
# 停止
nginx -s stop
# 重启
nginx -s reload

7.添加配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
server {
listen 80;
server_name xxx.xxx.xxx;

proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

location / {
proxy_pass http://127.0.0.1:8888;
proxy_connect_timeout 600;
proxy_read_timeout 600;
}
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import pandas as pd

# 生成DataFrame,以读取excel为例
excelFile = 'TEMP.xlsx'
df = pd.DataFrame(pd.read_excel(excelFile))

# 过滤字段
df1 = df[['Project Name', 'Status', 'Offer', 'Tax Rate', 'PM']]

# 条件筛选
df2 = df1.loc[df1['PM'] == 'Bob']

# 多条件筛选
df3 = df1.loc[df1['PM'] == 'Bob'].loc[df1['Status'] == 'Completed']

# 获取某列的值
pmList = df1[['PM']].values.T.tolist()[:][0]
print(pmList)

# 遍历
for pm in list(set(pmList)):
dfByPM = df1.loc[df1['PM'] == pm]
print('\r\n')
print(dfByPM)

# 统计总和
sum_list = [['PM', 'Offer']]
for pm in pmList:
temp = []
dfByPM = df.loc[df['PM'] == pm]
temp.append(pm)
for col in dfByPM.columns:
if col == 'Offer':
sumValue = dfByPM[col].sum() # 计数指定列的和
temp.append(sumValue)
sum_list.append(temp)

django + uwsgi 配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# uwsig使用配置文件启动
[uwsgi]
# 项目目录
chdir = /home/www/report/
# 指定项目的application
module = report.wsgi:application
# 指定sock的文件路径
socket = /home/www/report/conf/uwsgi.sock
# 进程个数
workers = 5
pidfile = /home/www/report/conf/uwsgi.pid
# 指定IP端口
http = 0.0.0.0:8888
# 指定静态文件
static-map = /static=/home/www/report/static
# 启动uwsgi的用户名和用户组
uid = root
gid = root
# 启用主进程
master = true
# 自动移除unix Socket和pid文件当服务停止的时候
vacuum = true
# 序列化接受的内容,如果可能的话
thunder-lock = true
# 启用线程
enable-threads = true
# 设置自中断时间
harakiri = 30
# 设置缓冲
post-buffering = 4096
# 设置日志目录
# daemonize = /home/www/report/conf/uwsgi.log
home = /root/.local/share/virtualenvs/report-kHgYRlIp

flask + uwsgi 配置

1
2
3
4
5
6
7
8
9
10
[uwsgi]
http = 0.0.0.0:8888
pythonpath = /home/ubuntu/www/index
wsgi-file = /home/ubuntu/www/index/app.py
callable = app
processes = 1
threads = 10
demonize = /home/ubuntu/www/index/log/serve.log
home = /home/ubuntu/.local/share/virtualenvs/index-1SLxCpRK

uwsgi 启动

1
2
3
4
5
6
# 启动(可以加-d守护进程启动)
uwsgi --ini xxx.ini
## 重启
uwsgi --reload xxx.pid
#### 停止:
uwsgi --stop xxx.pid