Compare commits
95 Commits
Author | SHA1 | Date |
---|---|---|
|
0c35e096c2 | |
|
142a36f608 | |
|
b18afb95c1 | |
|
aa87b24e1a | |
|
fdc51ba605 | |
|
ce6249142f | |
|
26c8b307d7 | |
|
6585b217bc | |
|
b609c729c3 | |
|
432ced4e0e | |
|
116116c8c4 | |
|
ca08bd7d6e | |
|
81610e505a | |
|
17439be724 | |
|
91999fbdcb | |
|
6645b00f24 | |
|
acb7b49551 | |
|
ba416d1496 | |
|
ff5e484997 | |
|
b900c28347 | |
|
5d17242ffc | |
|
39357660ab | |
|
49abb7c286 | |
|
8185feca1d | |
|
5789d4d774 | |
|
2fdf44d308 | |
|
5941bfb9bc | |
|
0c24dcb0a2 | |
|
a1d683a7c9 | |
|
be65db1892 | |
|
eb93fdd52a | |
|
e7e52acfa7 | |
|
02838da959 | |
|
0e73a0864c | |
|
61d347609e | |
|
a9df169b5a | |
|
98efe26906 | |
|
384a9cc1bb | |
|
0ffdca3e0b | |
|
5ae4071508 | |
|
4082efa1fc | |
|
fc03e6483c | |
|
2b8d29a0ab | |
|
881c11d51f | |
|
855922964e | |
|
9e814d2d43 | |
|
cdfbebf407 | |
|
017cacf12c | |
|
ab4a4bbb92 | |
|
853119b615 | |
|
f99398c649 | |
|
a61a79aca9 | |
|
00570523fa | |
|
508a912c35 | |
|
3ee0d33fff | |
|
03e4419157 | |
|
3998dfedf1 | |
|
1ffec6a785 | |
|
0fad6ef8cc | |
|
46b3590fb6 | |
|
a2d870e15e | |
|
758b18cb7d | |
|
d630fb16b4 | |
|
7c6f889e72 | |
|
0826dc9c15 | |
|
96111de3da | |
|
137fdee7f1 | |
|
c4200d7516 | |
|
30b331b4b0 | |
|
8fceffde07 | |
|
8dde539a79 | |
|
1cd2e3b408 | |
|
ee3040f011 | |
|
c5d574e357 | |
|
eb3e74a678 | |
|
1f16efb043 | |
|
467752e355 | |
|
91bf0d93f8 | |
|
9cab680e46 | |
|
c62e93a677 | |
|
e3354fefb0 | |
|
57e03c8aee | |
|
f1c1c00190 | |
|
4fced18886 | |
|
bdfa05670e | |
|
d389587d96 | |
|
e5283d2a23 | |
|
e65fba55a0 | |
|
b17f6fff8d | |
|
8748f0c2be | |
|
c38eed392d | |
|
070ac65f78 | |
|
a48db6ec63 | |
|
a6f8708aa0 | |
|
98fad6eb9d |
|
@ -11,3 +11,4 @@ target
|
|||
node_modules
|
||||
test/derby.log
|
||||
/console-fe
|
||||
.flattened-pom.xml
|
||||
|
|
41
README.md
41
README.md
|
@ -1,9 +1,11 @@
|
|||
# Nacos Sync
|
||||
|
||||
## [Example](https://github.com/paderlol/nacos-sync-example)
|
||||
|
||||
## Function
|
||||
|
||||
- Console: provide API and console for management
|
||||
- Worker: provider the service registration synchronization.
|
||||
- Worker: provide the service registration synchronization.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
@ -42,15 +44,9 @@ Info | +------------+ ^
|
|||
- NacosCluster target will dedup the synchronization information from Nacos.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Quick Start:
|
||||
- Swagger API: http://127.0.0.1:8081/swagger-ui.html#/
|
||||
- Web Console: http://127.0.0.1:8081/
|
||||
- Swagger API: http://127.0.0.1:8083/swagger-ui.html#/
|
||||
- Web Console: http://127.0.0.1:8083/
|
||||
- Others: TBD
|
||||
|
||||
# NacosSync Migration User Guide
|
||||
|
@ -82,7 +78,7 @@ Before you begin, install the following:
|
|||
|
||||
- 64bit OS: Linux/Unix/Mac/Windows supported, Linux/Unix/Mac recommended.
|
||||
- 64bit JDK 1.8+: downloads, JAVA_HOME settings.
|
||||
- Maven 3.2.x+: downloads, settings.
|
||||
- Maven 3.5.2+: [downloads](https://maven.apache.org/download.cgi), [settings](https://maven.apache.org/settings.html).
|
||||
- MySql 5.6.+
|
||||
|
||||
## Download & Build From Release
|
||||
|
@ -94,7 +90,7 @@ There are two ways to get NacosSync.
|
|||
|
||||
``` xml
|
||||
|
||||
cd nacosSync/
|
||||
cd nacos-sync/
|
||||
mvn clean package -U
|
||||
|
||||
```
|
||||
|
@ -103,7 +99,7 @@ The path to the target file:
|
|||
|
||||
``` xml
|
||||
|
||||
nacos-sync/nacossync-distribution/target/nacosSync.0.3.8.zip
|
||||
nacos-sync/nacossync-distribution/target/nacos-sync-0.5.0.tar.gz
|
||||
|
||||
```
|
||||
|
||||
|
@ -111,7 +107,7 @@ After extracting the installation package, the directory structure:
|
|||
|
||||
``` xml
|
||||
|
||||
nacosSync
|
||||
nacos-sync
|
||||
├── LICENSE
|
||||
├── NOTICE
|
||||
├── bin
|
||||
|
@ -122,7 +118,7 @@ nacosSync
|
|||
│ ├── application.properties
|
||||
│ └── logback-spring.xml
|
||||
├── logs
|
||||
└── nacosSync-server.jar
|
||||
└── nacos-sync-server.jar
|
||||
|
||||
```
|
||||
|
||||
|
@ -130,7 +126,7 @@ nacosSync
|
|||
|
||||
The default is Mysql database, which can support other relational databases
|
||||
|
||||
- Build db schema, the default schema name nacos_Sync.
|
||||
- Build db schema, the default schema name nacos_sync.
|
||||
- Tables do not need to be created separately, which is conducive to hibernate's automatic table creation function.
|
||||
- If the automatic table creation fails, you can build the table nacosSync.sql, the table statement is in the bin folder.
|
||||
|
||||
|
@ -159,15 +155,18 @@ sh startup.sh start
|
|||
|
||||
``` xml
|
||||
|
||||
http://127.0.0.1:8081/#/serviceSync
|
||||
http://127.0.0.1:8083/#/serviceSync
|
||||
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Full Synchronization from Zookeeper to Nacos (Dubbo)
|
||||
When “*” is entered in the “Service Name” field of this form, it will fully synchronize all services from Zookeeper to Nacos, but only when using Dubbo.
|
||||

|
||||
|
||||
### Full Synchronization from Nacos to Nacos
|
||||
When “All” is entered in the “Service Name” field of this form, it will automatically synchronize all registered services within the **default group** of the current cluster.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
This description explains the functionality clearly for English-speaking users.
|
||||

|
|
@ -16,7 +16,7 @@
|
|||
<parent>
|
||||
<artifactId>nacossync-parent</artifactId>
|
||||
<groupId>com.alibaba.nacossync</groupId>
|
||||
<version>0.4.4</version>
|
||||
<version>${revision}</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
|
@ -27,5 +27,4 @@
|
|||
<artifactId>junit</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
</project>
|
||||
</project>
|
File diff suppressed because one or more lines are too long
|
@ -7,8 +7,8 @@
|
|||
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
||||
<title>Nacos-Sync</title>
|
||||
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
||||
<link href="./css/main.68bce23a.css" rel="stylesheet"></head>
|
||||
<link href="./css/main.e2917886.css" rel="stylesheet"></head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="text/javascript" src="./js/main.1cd0c600.js"></script></body>
|
||||
<script type="text/javascript" src="./js/main.b73436b5.js"></script></body>
|
||||
</html>
|
||||
|
|
File diff suppressed because one or more lines are too long
|
@ -18,6 +18,9 @@ class AddConfigDialog extends React.Component {
|
|||
visible: false,
|
||||
clusterName: '',
|
||||
clusterType: '',
|
||||
namespace: '',
|
||||
password: '',
|
||||
userName: '',
|
||||
connectKeyList: [],
|
||||
};
|
||||
}
|
||||
|
@ -27,8 +30,8 @@ class AddConfigDialog extends React.Component {
|
|||
}
|
||||
|
||||
save() {
|
||||
const { clusterName, clusterType, connectKeyList } = this.state;
|
||||
add({ clusterName, clusterType, connectKeyList })
|
||||
const { clusterName, namespace, userName, password, clusterType, connectKeyList } = this.state;
|
||||
add({ clusterName, namespace, userName, password, clusterType, connectKeyList })
|
||||
.then(() => {
|
||||
this.props.turnPage(1);
|
||||
this.close();
|
||||
|
@ -37,7 +40,10 @@ class AddConfigDialog extends React.Component {
|
|||
}
|
||||
|
||||
close() {
|
||||
this.setState({ visible: false });
|
||||
this.setState({
|
||||
visible: false,
|
||||
clusterType: '',
|
||||
});
|
||||
}
|
||||
|
||||
open = () => this.setState({ visible: true })
|
||||
|
@ -71,6 +77,35 @@ class AddConfigDialog extends React.Component {
|
|||
}
|
||||
</Select>
|
||||
</FormItem>
|
||||
{
|
||||
this.state.clusterType === 'NACOS' && (
|
||||
<>
|
||||
<FormItem
|
||||
label={`${locale.namespace}:`}
|
||||
>
|
||||
<Input
|
||||
placeholder={locale.namespacePlaceholder}
|
||||
onChange={ns => this.setState({ namespace: ns })}
|
||||
/>
|
||||
</FormItem>
|
||||
<FormItem
|
||||
label={`${locale.username}:`}
|
||||
>
|
||||
<Input
|
||||
placeholder={locale.usernamePlaceholder}
|
||||
onChange={un => this.setState({ userName: un })}
|
||||
/>
|
||||
</FormItem>
|
||||
<FormItem
|
||||
label={`${locale.password}:`}
|
||||
>
|
||||
<Input.Password
|
||||
placeholder={locale.passwordPlaceholder}
|
||||
onChange={pw => this.setState({ password: pw })}
|
||||
/>
|
||||
</FormItem>
|
||||
</>)
|
||||
}
|
||||
<FormItem label={`${locale.connectKeyList}:`}>
|
||||
<Input.TextArea
|
||||
onChange={(connectKeyListStr) => {
|
||||
|
|
|
@ -98,6 +98,7 @@ class ClusterConfig extends React.Component {
|
|||
<Table.Column title={locale.clusterName} dataIndex="clusterName" />
|
||||
<Table.Column title={locale.clusterType} dataIndex="clusterType" />
|
||||
<Table.Column title={locale.connectKeyList} dataIndex="connectKeyList" />
|
||||
<Table.Column title={locale.namespace} dataIndex="namespace" />
|
||||
<Table.Column
|
||||
title={locale.operation}
|
||||
cell={(value, index, record) => (
|
||||
|
|
|
@ -13,7 +13,7 @@ class Layout extends React.Component {
|
|||
<Header />
|
||||
<Row className="layout">
|
||||
<Col fixedSpan="9" className="nav-bar">
|
||||
<h1 className="title">Nacos-Sync 3.0</h1>
|
||||
<h1 className="title">Nacos-Sync 0.4.8</h1>
|
||||
<Menu />
|
||||
</Col>
|
||||
<Col className="main-panel">{this.props.children}</Col>
|
||||
|
|
|
@ -19,6 +19,9 @@ const I18N_CONF = {
|
|||
clusterNamePlaceholder: 'Please enter the cluster name',
|
||||
clusterType: 'Cluster Type',
|
||||
connectKeyList: 'Connect Key List',
|
||||
namespace: 'Namespace',
|
||||
password: 'Password',
|
||||
username: 'Username',
|
||||
operation: 'Operation',
|
||||
deleteBtn: 'Delete',
|
||||
confirm: 'Prompt',
|
||||
|
@ -29,6 +32,8 @@ const I18N_CONF = {
|
|||
title: 'New Cluster',
|
||||
clusterName: 'Cluster Name',
|
||||
clusterNamePlaceholder: 'Please enter the cluster name',
|
||||
namespace: 'Namespace',
|
||||
namespacePlaceholder: 'Please enter the namespace',
|
||||
clusterType: 'Cluster Type',
|
||||
connectKeyList: 'Connect IP',
|
||||
connectKeyListPlaceholder: 'Please enter the ip',
|
||||
|
|
|
@ -19,6 +19,7 @@ const I18N_CONF = {
|
|||
clusterNamePlaceholder: '请输入集群名',
|
||||
clusterType: '集群类型',
|
||||
connectKeyList: '集群IP列表',
|
||||
namespace: '命名空间',
|
||||
operation: '操作',
|
||||
deleteBtn: '删除',
|
||||
confirm: '提示',
|
||||
|
@ -29,6 +30,12 @@ const I18N_CONF = {
|
|||
title: '新增集群',
|
||||
clusterName: '集群名',
|
||||
clusterNamePlaceholder: '请输入集群名',
|
||||
namespace: '命名空间',
|
||||
namespacePlaceholder: '请输入命名空间',
|
||||
password: '密码',
|
||||
passwordPlaceholder: '请输入密码',
|
||||
username: '用户名',
|
||||
usernamePlaceholder: '请输入用户名',
|
||||
clusterType: '集群类型',
|
||||
connectKeyList: '集群IP列表',
|
||||
connectKeyListPlaceholder: '请输入集群IP',
|
||||
|
|
File diff suppressed because one or more lines are too long
|
@ -7,8 +7,8 @@
|
|||
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
||||
<title>Nacos-Sync</title>
|
||||
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
||||
<link href="./css/main.68bce23a.css" rel="stylesheet"></head>
|
||||
<link href="./css/main.e2917886.css" rel="stylesheet"></head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="text/javascript" src="./js/main.1cd0c600.js"></script></body>
|
||||
<script type="text/javascript" src="./js/main.b73436b5.js"></script></body>
|
||||
</html>
|
||||
|
|
File diff suppressed because one or more lines are too long
|
@ -1,5 +1,5 @@
|
|||
/******************************************/
|
||||
/* DB name = nacos_Sync */
|
||||
/* DB name = nacos_sync */
|
||||
/* Table name = cluster */
|
||||
/******************************************/
|
||||
CREATE TABLE `cluster` (
|
||||
|
@ -8,10 +8,14 @@ CREATE TABLE `cluster` (
|
|||
`cluster_name` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`cluster_type` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`connect_key_list` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`user_name` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`password` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`namespace` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`cluster_level` int default 0,
|
||||
PRIMARY KEY (`id`)
|
||||
) ENGINE=MyISAM AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||
/******************************************/
|
||||
/* DB name = nacos_Sync */
|
||||
/* DB name = nacos_sync */
|
||||
/* Table name = system_config */
|
||||
/******************************************/
|
||||
CREATE TABLE `system_config` (
|
||||
|
@ -20,9 +24,9 @@ CREATE TABLE `system_config` (
|
|||
`config_key` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`config_value` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
PRIMARY KEY (`id`)
|
||||
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||
/******************************************/
|
||||
/* DB name = nacos_Sync */
|
||||
/* DB name = nacos_sync */
|
||||
/* Table name = task */
|
||||
/******************************************/
|
||||
CREATE TABLE `task` (
|
||||
|
@ -37,5 +41,6 @@ CREATE TABLE `task` (
|
|||
`task_status` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`version` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`worker_ip` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||
`status` int default null ,
|
||||
PRIMARY KEY (`id`)
|
||||
) ENGINE=MyISAM AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||
|
|
|
@ -13,14 +13,14 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
pid=`ps ax | grep -i 'nacosSync' |grep java | grep -v grep | awk '{print $1}'`
|
||||
pid=`ps ax | grep -i 'nacos-sync' |grep java | grep -v grep | awk '{print $1}'`
|
||||
if [ -z "$pid" ] ; then
|
||||
echo "No nacosSync running."
|
||||
echo "no nacos-sync running."
|
||||
exit -1;
|
||||
fi
|
||||
|
||||
echo "The nacosSync(${pid}) is running..."
|
||||
echo "the nacos-sync(${pid}) is running..."
|
||||
|
||||
kill ${pid}
|
||||
|
||||
echo "Send shutdown request to nacosSync(${pid}) OK"
|
||||
echo "Send shutdown request to nacos-sync(${pid}) OK"
|
|
@ -0,0 +1 @@
|
|||
java -jar ../nacos-sync-server.jar -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m --spring.config.location=../conf/application.properties
|
|
@ -47,24 +47,24 @@ export BASE_DIR=`cd $(dirname $0)/..; pwd`
|
|||
|
||||
|
||||
JAVA_OPT="${JAVA_OPT} -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m"
|
||||
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${BASE_DIR}/nacossync_java_heapdump.hprof"
|
||||
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${BASE_DIR}/nacos-sync-java-heapdump.hprof"
|
||||
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"
|
||||
JAVA_OPT="${JAVA_OPT} -Dspring.config.location=${BASE_DIR}/conf/application.properties"
|
||||
JAVA_OPT="${JAVA_OPT} -DnacosSync.home=${BASE_DIR}"
|
||||
|
||||
JAVA_MAJOR_VERSION=$($JAVA -version 2>&1 | sed -E -n 's/.* version "([0-9]*).*$/\1/p')
|
||||
if [[ "$JAVA_MAJOR_VERSION" -ge "9" ]] ; then
|
||||
JAVA_OPT="${JAVA_OPT} -Xlog:gc*:file=${BASE_DIR}/logs/nacossync_gc.log:time,tags:filecount=10,filesize=102400"
|
||||
JAVA_OPT="${JAVA_OPT} -Xlog:gc*:file=${BASE_DIR}/logs/nacos-sync-gc.log:time,tags:filecount=10,filesize=102400"
|
||||
else
|
||||
JAVA_OPT="${JAVA_OPT} -Xloggc:${BASE_DIR}/logs/nacossync_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M"
|
||||
JAVA_OPT="${JAVA_OPT} -Xloggc:${BASE_DIR}/logs/nacos-sync-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M"
|
||||
fi
|
||||
|
||||
JAVA_OPT="${JAVA_OPT} -jar ${BASE_DIR}/nacosSync-server.jar"
|
||||
JAVA_OPT="${JAVA_OPT} -jar ${BASE_DIR}/nacos-sync-server.jar"
|
||||
JAVA_OPT="${JAVA_OPT} --logging.config=${BASE_DIR}/conf/logback-spring.xml"
|
||||
|
||||
echo "=============JAVA_HOME:"$JAVA_HOME
|
||||
echo "=============BASE_DIR:"$BASE_DIR
|
||||
echo "=============JAVA:"$JAVA
|
||||
echo "JAVA_HOME:"$JAVA_HOME
|
||||
echo "BASE_DIR:"$BASE_DIR
|
||||
echo "JAVA:"$JAVA
|
||||
|
||||
|
||||
if [ ! -d "${BASE_DIR}/logs" ]; then
|
||||
|
@ -78,9 +78,9 @@ usage(){
|
|||
|
||||
start(){
|
||||
|
||||
echo "$JAVA ${JAVA_OPT}" > ${BASE_DIR}/logs/nacossync_start.out 2>&1 &
|
||||
nohup $JAVA ${JAVA_OPT} >> ${BASE_DIR}/logs/nacossync_start.out 2>&1 &
|
||||
echo "nacossync is starting,you can check the ${BASE_DIR}/logs/nacossync_start.out"
|
||||
echo "$JAVA ${JAVA_OPT}" > ${BASE_DIR}/logs/nacos-sync-start.out 2>&1 &
|
||||
nohup $JAVA ${JAVA_OPT} >> ${BASE_DIR}/logs/nacos-sync-start.out 2>&1 &
|
||||
echo "nacos-sync is starting,you can check the ${BASE_DIR}/logs/nacos-sync-start.out"
|
||||
|
||||
}
|
||||
|
||||
|
|
|
@ -5,14 +5,14 @@
|
|||
<parent>
|
||||
<artifactId>nacossync-parent</artifactId>
|
||||
<groupId>com.alibaba.nacossync</groupId>
|
||||
<version>0.4.4</version>
|
||||
<version>${revision}</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<packaging>pom</packaging>
|
||||
<artifactId>nacossync-distribution</artifactId>
|
||||
|
||||
<build>
|
||||
<finalName>nacosSync.${parent.version}</finalName>
|
||||
<finalName>nacos-sync-${project.version}</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<artifactId>maven-assembly-plugin</artifactId>
|
||||
|
@ -35,4 +35,4 @@
|
|||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
</project>
|
|
@ -17,7 +17,7 @@
|
|||
<id>bin</id>
|
||||
<includeBaseDirectory>true</includeBaseDirectory>
|
||||
<!-- file name after unzip -->
|
||||
<baseDirectory>nacosSync</baseDirectory>
|
||||
<baseDirectory>nacos-sync</baseDirectory>
|
||||
<formats>
|
||||
<format>dir</format>
|
||||
<format>tar.gz</format>
|
||||
|
@ -60,9 +60,9 @@
|
|||
<destName>NOTICE</destName>
|
||||
</file>
|
||||
<file>
|
||||
<source>../nacossync-worker/target/nacosSync-server.${parent.version}.jar</source>
|
||||
<source>../nacossync-worker/target/nacos-sync-server-${project.version}.jar</source>
|
||||
<outputDirectory></outputDirectory>
|
||||
<destName>nacosSync-server.jar</destName>
|
||||
<destName>nacos-sync-server.jar</destName>
|
||||
</file>
|
||||
</files>
|
||||
</assembly>
|
||||
|
|
|
@ -17,7 +17,7 @@
|
|||
<parent>
|
||||
<artifactId>nacossync-parent</artifactId>
|
||||
<groupId>com.alibaba.nacossync</groupId>
|
||||
<version>0.4.4</version>
|
||||
<version>${revision}</version>
|
||||
<relativePath>../pom.xml</relativePath>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
@ -25,27 +25,19 @@
|
|||
|
||||
<artifactId>nacossync-test</artifactId>
|
||||
|
||||
<properties>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
</properties>
|
||||
<dependencies>
|
||||
|
||||
<dependency>
|
||||
<groupId>io.projectreactor</groupId>
|
||||
<artifactId>reactor-core</artifactId>
|
||||
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.alibaba.nacossync</groupId>
|
||||
<groupId>${project.groupId}</groupId>
|
||||
<artifactId>nacossync-worker</artifactId>
|
||||
<version>0.4.4</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-test</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework</groupId>
|
||||
<artifactId>spring-test</artifactId>
|
||||
|
@ -57,10 +49,8 @@
|
|||
<dependency>
|
||||
<groupId>com.ning</groupId>
|
||||
<artifactId>async-http-client</artifactId>
|
||||
<version>1.7.17</version>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
|
@ -68,7 +58,6 @@
|
|||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-surefire-plugin</artifactId>
|
||||
<version>2.9</version>
|
||||
<configuration>
|
||||
<parallel>methods</parallel>
|
||||
<useUnlimitedThreads>true</useUnlimitedThreads>
|
||||
|
@ -76,6 +65,4 @@
|
|||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
|
||||
</project>
|
||||
</project>
|
|
@ -16,29 +16,11 @@
|
|||
<parent>
|
||||
<artifactId>nacossync-parent</artifactId>
|
||||
<groupId>com.alibaba.nacossync</groupId>
|
||||
<version>0.4.4</version>
|
||||
<version>${revision}</version>
|
||||
</parent>
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<artifactId>nacossync-worker</artifactId>
|
||||
<version>0.4.4</version>
|
||||
<properties>
|
||||
<zookeeper.version>3.4.9</zookeeper.version>
|
||||
<curator.version>4.1.0</curator.version>
|
||||
<cloud.version>Finchley.SR2</cloud.version>
|
||||
<mockito.version>1.10.19</mockito.version>
|
||||
<nacos.client.verison>1.3.1</nacos.client.verison>
|
||||
</properties>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.cloud</groupId>
|
||||
<artifactId>spring-cloud-dependencies</artifactId>
|
||||
<version>${cloud.version}</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
|
@ -52,10 +34,6 @@
|
|||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-web</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-logging</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>junit</groupId>
|
||||
<artifactId>junit</artifactId>
|
||||
|
@ -64,12 +42,12 @@
|
|||
<dependency>
|
||||
<groupId>org.mockito</groupId>
|
||||
<artifactId>mockito-all</artifactId>
|
||||
<version>${mockito.version}</version>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.projectlombok</groupId>
|
||||
<artifactId>lombok</artifactId>
|
||||
<scope>provided</scope>
|
||||
</dependency>
|
||||
<!-- 默认使用HikariCP连接池 -->
|
||||
<dependency>
|
||||
|
@ -82,43 +60,36 @@
|
|||
</dependency>
|
||||
<!-- mysql -->
|
||||
<dependency>
|
||||
<groupId>mysql</groupId>
|
||||
<artifactId>mysql-connector-java</artifactId>
|
||||
<groupId>com.mysql</groupId>
|
||||
<artifactId>mysql-connector-j</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.alibaba</groupId>
|
||||
<artifactId>fastjson</artifactId>
|
||||
<version>1.2.47</version>
|
||||
</dependency>
|
||||
<!-- swagger2 -->
|
||||
<dependency>
|
||||
<groupId>io.springfox</groupId>
|
||||
<artifactId>springfox-swagger2</artifactId>
|
||||
<version>2.6.1</version>
|
||||
<groupId>org.springdoc</groupId>
|
||||
<artifactId>springdoc-openapi-ui</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.springfox</groupId>
|
||||
<artifactId>springfox-swagger-ui</artifactId>
|
||||
<version>2.6.1</version>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
</dependency>
|
||||
<!--nacos-->
|
||||
<dependency>
|
||||
<groupId>com.alibaba.nacos</groupId>
|
||||
<artifactId>nacos-client</artifactId>
|
||||
<version>${nacos.client.verison}</version>
|
||||
</dependency>
|
||||
|
||||
<!--zookeeper-->
|
||||
<dependency>
|
||||
<groupId>org.apache.zookeeper</groupId>
|
||||
<artifactId>zookeeper</artifactId>
|
||||
<version>${zookeeper.version}</version>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<artifactId>slf4j-log4j12</artifactId>
|
||||
<groupId>org.slf4j</groupId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.apache.curator</groupId>
|
||||
<artifactId>curator-recipes</artifactId>
|
||||
<version>${curator.version}</version>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<groupId>org.apache.zookeeper</groupId>
|
||||
|
@ -126,22 +97,42 @@
|
|||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.apache.curator</groupId>
|
||||
<artifactId>curator-framework</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.curator</groupId>
|
||||
<artifactId>curator-client</artifactId>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<artifactId>slf4j-api</artifactId>
|
||||
<groupId>org.slf4j</groupId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
<!-- eureka -->
|
||||
<dependency>
|
||||
<groupId>org.springframework.cloud</groupId>
|
||||
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- consul -->
|
||||
<dependency>
|
||||
<groupId>com.ecwid.consul</groupId>
|
||||
<artifactId>consul-api</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.apache.commons</groupId>
|
||||
<artifactId>commons-lang3</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.google.guava</groupId>
|
||||
<artifactId>guava</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>nacosSync-server.${parent.version}</finalName>
|
||||
<finalName>nacos-sync-server-${project.version}</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
|
@ -170,6 +161,22 @@
|
|||
</archive>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-compiler-plugin</artifactId>
|
||||
<configuration>
|
||||
<source>17</source>
|
||||
<target>17</target>
|
||||
<annotationProcessorPaths>
|
||||
<path>
|
||||
<groupId>org.projectlombok</groupId>
|
||||
<artifactId>lombok</artifactId>
|
||||
<version>1.18.34</version>
|
||||
</path>
|
||||
</annotationProcessorPaths>
|
||||
</configuration>
|
||||
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
|
|
|
@ -14,24 +14,32 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync;
|
||||
|
||||
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
|
||||
import org.springframework.cloud.netflix.eureka.EurekaClientAutoConfiguration;
|
||||
import springfox.documentation.swagger2.annotations.EnableSwagger2;
|
||||
import org.springframework.context.ConfigurableApplicationContext;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: SkyWalkerMain.java, v 0.1 2018-09-24 PM12:42 NacosSync Exp $$
|
||||
*/
|
||||
@EnableSwagger2
|
||||
@SpringBootApplication(exclude = EurekaClientAutoConfiguration.class)
|
||||
public class NacosSyncMain {
|
||||
|
||||
|
||||
public static void main(String[] args) {
|
||||
|
||||
SpringApplication.run(NacosSyncMain.class, args);
|
||||
|
||||
ConfigurableApplicationContext context = SpringApplication.run(NacosSyncMain.class, args);
|
||||
|
||||
// Register shutdown callback using Spring Boot's context lifecycle
|
||||
context.registerShutdownHook();
|
||||
context.addApplicationListener(event -> {
|
||||
if (event instanceof org.springframework.context.event.ContextClosedEvent) {
|
||||
BatchTaskExecutor.shutdown();
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,6 +14,7 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.api;
|
||||
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
|
@ -44,57 +45,54 @@ import org.springframework.web.bind.annotation.RestController;
|
|||
@Slf4j
|
||||
@RestController
|
||||
public class ClusterApi {
|
||||
|
||||
|
||||
private final ClusterAddProcessor clusterAddProcessor;
|
||||
|
||||
|
||||
private final ClusterDeleteProcessor clusterDeleteProcessor;
|
||||
|
||||
|
||||
private final ClusterDetailQueryProcessor clusterDetailQueryProcessor;
|
||||
|
||||
|
||||
private final ClusterListQueryProcessor clusterListQueryProcessor;
|
||||
|
||||
public ClusterApi(
|
||||
ClusterAddProcessor clusterAddProcessor, ClusterDeleteProcessor clusterDeleteProcessor,
|
||||
ClusterDetailQueryProcessor clusterDetailQueryProcessor, ClusterListQueryProcessor clusterListQueryProcessor) {
|
||||
|
||||
public ClusterApi(ClusterAddProcessor clusterAddProcessor, ClusterDeleteProcessor clusterDeleteProcessor,
|
||||
ClusterDetailQueryProcessor clusterDetailQueryProcessor,
|
||||
ClusterListQueryProcessor clusterListQueryProcessor) {
|
||||
this.clusterAddProcessor = clusterAddProcessor;
|
||||
this.clusterDeleteProcessor = clusterDeleteProcessor;
|
||||
this.clusterDetailQueryProcessor = clusterDetailQueryProcessor;
|
||||
this.clusterListQueryProcessor = clusterListQueryProcessor;
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/cluster/list", method = RequestMethod.GET)
|
||||
public ClusterListQueryResult clusters(ClusterListQueryRequest clusterListQueryRequest) {
|
||||
|
||||
return SkyWalkerTemplate.run(clusterListQueryProcessor, clusterListQueryRequest,
|
||||
new ClusterListQueryResult());
|
||||
|
||||
return SkyWalkerTemplate.run(clusterListQueryProcessor, clusterListQueryRequest, new ClusterListQueryResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/cluster/detail", method = RequestMethod.GET)
|
||||
public ClusterDetailQueryResult getByTaskId(ClusterDetailQueryRequest clusterDetailQueryRequest) {
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(clusterDetailQueryProcessor, clusterDetailQueryRequest,
|
||||
new ClusterDetailQueryResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/cluster/delete", method = RequestMethod.DELETE)
|
||||
public ClusterDeleteResult deleteCluster(ClusterDeleteRequest clusterDeleteRequest) {
|
||||
|
||||
return SkyWalkerTemplate.run(clusterDeleteProcessor, clusterDeleteRequest,
|
||||
new ClusterDeleteResult());
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(clusterDeleteProcessor, clusterDeleteRequest, new ClusterDeleteResult());
|
||||
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/cluster/add", method = RequestMethod.POST)
|
||||
public ClusterAddResult clusterAdd(@RequestBody ClusterAddRequest clusterAddRequest) {
|
||||
|
||||
return SkyWalkerTemplate
|
||||
.run(clusterAddProcessor, clusterAddRequest, new ClusterAddResult());
|
||||
|
||||
return SkyWalkerTemplate.run(clusterAddProcessor, clusterAddRequest, new ClusterAddResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/cluster/types", method = RequestMethod.GET)
|
||||
public ClusterTypeResult getClusterType() {
|
||||
|
||||
|
||||
return new ClusterTypeResult(ClusterTypeEnum.getClusterTypeCodes());
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -14,6 +14,7 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.api;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
@ -42,34 +43,36 @@ import com.alibaba.nacossync.template.processor.ConfigQueryProcessor;
|
|||
@Slf4j
|
||||
@RestController
|
||||
public class SystemConfigApi {
|
||||
|
||||
@Autowired
|
||||
private ConfigQueryProcessor configQueryProcessor;
|
||||
|
||||
@Autowired
|
||||
private ConfigDeleteProcessor configDeleteProcessor;
|
||||
|
||||
@Autowired
|
||||
private ConfigAddProcessor configAddProcessor;
|
||||
|
||||
|
||||
private final ConfigQueryProcessor configQueryProcessor;
|
||||
|
||||
private final ConfigDeleteProcessor configDeleteProcessor;
|
||||
|
||||
private final ConfigAddProcessor configAddProcessor;
|
||||
|
||||
public SystemConfigApi(ConfigQueryProcessor configQueryProcessor, ConfigDeleteProcessor configDeleteProcessor,
|
||||
ConfigAddProcessor configAddProcessor) {
|
||||
this.configQueryProcessor = configQueryProcessor;
|
||||
this.configDeleteProcessor = configDeleteProcessor;
|
||||
this.configAddProcessor = configAddProcessor;
|
||||
}
|
||||
|
||||
@RequestMapping(path = "/v1/systemconfig/list", method = RequestMethod.GET)
|
||||
public ConfigQueryResult tasks(ConfigQueryRequest configQueryRequest) {
|
||||
|
||||
return SkyWalkerTemplate.run(configQueryProcessor, configQueryRequest,
|
||||
new ConfigQueryResult());
|
||||
|
||||
return SkyWalkerTemplate.run(configQueryProcessor, configQueryRequest, new ConfigQueryResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/systemconfig/delete", method = RequestMethod.DELETE)
|
||||
public ConfigDeleteResult deleteTask(@RequestBody ConfigDeleteRequest configDeleteRequest) {
|
||||
|
||||
return SkyWalkerTemplate.run(configDeleteProcessor, configDeleteRequest,
|
||||
new ConfigDeleteResult());
|
||||
|
||||
return SkyWalkerTemplate.run(configDeleteProcessor, configDeleteRequest, new ConfigDeleteResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/systemconfig/add", method = RequestMethod.POST)
|
||||
public ConfigAddResult taskAdd(@RequestBody ConfigAddRequest configAddRequest) {
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(configAddProcessor, configAddRequest, new ConfigAddResult());
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -10,8 +10,10 @@
|
|||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.api;
|
||||
|
||||
import com.alibaba.nacossync.pojo.request.TaskAddAllRequest;
|
||||
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
||||
import com.alibaba.nacossync.pojo.request.TaskDeleteInBatchRequest;
|
||||
import com.alibaba.nacossync.pojo.request.TaskDeleteRequest;
|
||||
|
@ -23,6 +25,7 @@ import com.alibaba.nacossync.pojo.result.TaskAddResult;
|
|||
import com.alibaba.nacossync.pojo.result.TaskDetailQueryResult;
|
||||
import com.alibaba.nacossync.pojo.result.TaskListQueryResult;
|
||||
import com.alibaba.nacossync.template.SkyWalkerTemplate;
|
||||
import com.alibaba.nacossync.template.processor.TaskAddAllProcessor;
|
||||
import com.alibaba.nacossync.template.processor.TaskAddProcessor;
|
||||
import com.alibaba.nacossync.template.processor.TaskDeleteInBatchProcessor;
|
||||
import com.alibaba.nacossync.template.processor.TaskDeleteProcessor;
|
||||
|
@ -42,67 +45,83 @@ import org.springframework.web.bind.annotation.RestController;
|
|||
@Slf4j
|
||||
@RestController
|
||||
public class TaskApi {
|
||||
|
||||
|
||||
private final TaskUpdateProcessor taskUpdateProcessor;
|
||||
|
||||
|
||||
private final TaskAddProcessor taskAddProcessor;
|
||||
|
||||
|
||||
private final TaskAddAllProcessor taskAddAllProcessor;
|
||||
|
||||
private final TaskDeleteProcessor taskDeleteProcessor;
|
||||
|
||||
|
||||
private final TaskDeleteInBatchProcessor taskDeleteInBatchProcessor;
|
||||
|
||||
|
||||
private final TaskListQueryProcessor taskListQueryProcessor;
|
||||
|
||||
|
||||
private final TaskDetailProcessor taskDetailProcessor;
|
||||
|
||||
|
||||
public TaskApi(TaskUpdateProcessor taskUpdateProcessor, TaskAddProcessor taskAddProcessor,
|
||||
TaskDeleteProcessor taskDeleteProcessor, TaskDeleteInBatchProcessor taskDeleteInBatchProcessor,
|
||||
TaskListQueryProcessor taskListQueryProcessor, TaskDetailProcessor taskDetailProcessor) {
|
||||
TaskAddAllProcessor taskAddAllProcessor, TaskDeleteProcessor taskDeleteProcessor,
|
||||
TaskDeleteInBatchProcessor taskDeleteInBatchProcessor, TaskListQueryProcessor taskListQueryProcessor,
|
||||
TaskDetailProcessor taskDetailProcessor) {
|
||||
this.taskUpdateProcessor = taskUpdateProcessor;
|
||||
this.taskAddProcessor = taskAddProcessor;
|
||||
this.taskAddAllProcessor = taskAddAllProcessor;
|
||||
this.taskDeleteProcessor = taskDeleteProcessor;
|
||||
this.taskDeleteInBatchProcessor = taskDeleteInBatchProcessor;
|
||||
this.taskListQueryProcessor = taskListQueryProcessor;
|
||||
this.taskDetailProcessor = taskDetailProcessor;
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/task/list", method = RequestMethod.GET)
|
||||
public TaskListQueryResult tasks(TaskListQueryRequest taskListQueryRequest) {
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(taskListQueryProcessor, taskListQueryRequest, new TaskListQueryResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/task/detail", method = RequestMethod.GET)
|
||||
public TaskDetailQueryResult getByTaskId(TaskDetailQueryRequest taskDetailQueryRequest) {
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(taskDetailProcessor, taskDetailQueryRequest, new TaskDetailQueryResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/task/delete", method = RequestMethod.DELETE)
|
||||
public BaseResult deleteTask(TaskDeleteRequest taskDeleteRequest) {
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(taskDeleteProcessor, taskDeleteRequest, new BaseResult());
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* @author yongchao9
|
||||
* @param taskBatchDeleteRequest
|
||||
* @return
|
||||
* @author yongchao9
|
||||
*/
|
||||
@RequestMapping(path = "/v1/task/deleteInBatch", method = RequestMethod.DELETE)
|
||||
public BaseResult batchDeleteTask(TaskDeleteInBatchRequest taskBatchDeleteRequest) {
|
||||
return SkyWalkerTemplate.run(taskDeleteInBatchProcessor, taskBatchDeleteRequest, new BaseResult());
|
||||
}
|
||||
|
||||
|
||||
@RequestMapping(path = "/v1/task/add", method = RequestMethod.POST)
|
||||
public BaseResult taskAdd(@RequestBody TaskAddRequest addTaskRequest) {
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(taskAddProcessor, addTaskRequest, new TaskAddResult());
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* TODO 目前仅支持 Nacos 为源的同步类型,待完善更多类型支持.
|
||||
* <p>
|
||||
* 支持从 sourceCluster 获取所有 service,然后生成同步到 destCluster 的任务。
|
||||
* </p>
|
||||
*/
|
||||
@RequestMapping(path = "/v1/task/addAll", method = RequestMethod.POST)
|
||||
public BaseResult taskAddAll(@RequestBody TaskAddAllRequest addAllRequest) {
|
||||
|
||||
return SkyWalkerTemplate.run(taskAddAllProcessor, addAllRequest, new TaskAddResult());
|
||||
}
|
||||
|
||||
@RequestMapping(path = "/v1/task/update", method = RequestMethod.POST)
|
||||
public BaseResult updateTask(@RequestBody TaskUpdateRequest taskUpdateRequest) {
|
||||
|
||||
|
||||
return SkyWalkerTemplate.run(taskUpdateProcessor, taskUpdateRequest, new BaseResult());
|
||||
}
|
||||
}
|
||||
|
|
|
@ -16,8 +16,7 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.cache;
|
||||
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import com.alibaba.fastjson.TypeReference;
|
||||
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||
|
@ -25,39 +24,46 @@ import com.alibaba.nacossync.pojo.FinishedTask;
|
|||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import org.jboss.netty.util.internal.ThreadLocalRandom;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import lombok.SneakyThrows;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.util.CollectionUtils;
|
||||
import org.springframework.util.StringUtils;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.ThreadLocalRandom;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: SkyWalkerCacheServices.java, v 0.1 2018-09-27 AM2:47 NacosSync Exp $$
|
||||
*/
|
||||
@Service
|
||||
public class SkyWalkerCacheServices {
|
||||
|
||||
private static final Map<String, FinishedTask> FINISHED_TASK_MAP = new ConcurrentHashMap<>();
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
@Autowired
|
||||
private ClusterAccessService clusterAccessService;
|
||||
|
||||
private static Map<String, FinishedTask> finishedTaskMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public SkyWalkerCacheServices(ClusterAccessService clusterAccessService, ObjectMapper objectMapper) {
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
this.objectMapper = objectMapper;
|
||||
}
|
||||
|
||||
public String getClusterConnectKey(String clusterId) {
|
||||
List<String> allClusterConnectKey = getAllClusterConnectKey(clusterId);
|
||||
|
||||
return allClusterConnectKey.get(ThreadLocalRandom.current().nextInt(allClusterConnectKey.size()));
|
||||
}
|
||||
|
||||
@SneakyThrows
|
||||
public List<String> getAllClusterConnectKey(String clusterId) {
|
||||
ClusterDO clusterDO = clusterAccessService.findByClusterId(clusterId);
|
||||
|
||||
List<String> connectKeyList = JSONObject.parseObject(clusterDO.getConnectKeyList(),
|
||||
new TypeReference<List<String>>() {
|
||||
});
|
||||
List<String> connectKeyList = objectMapper.readerForListOf(String.class)
|
||||
.readValue(clusterDO.getConnectKeyList());
|
||||
|
||||
if (CollectionUtils.isEmpty(connectKeyList)) {
|
||||
throw new SkyWalkerException("getClusterConnectKey empty, clusterId:" + clusterId);
|
||||
|
@ -79,23 +85,32 @@ public class SkyWalkerCacheServices {
|
|||
FinishedTask finishedTask = new FinishedTask();
|
||||
finishedTask.setOperationId(operationId);
|
||||
|
||||
finishedTaskMap.put(operationId, finishedTask);
|
||||
FINISHED_TASK_MAP.put(operationId, finishedTask);
|
||||
}
|
||||
|
||||
public FinishedTask getFinishedTask(TaskDO taskDO) {
|
||||
|
||||
String operationId = SkyWalkerUtil.getOperationId(taskDO);
|
||||
|
||||
if (StringUtils.isEmpty(operationId)) {
|
||||
if (!StringUtils.hasLength(operationId)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return finishedTaskMap.get(operationId);
|
||||
return FINISHED_TASK_MAP.get(operationId);
|
||||
}
|
||||
|
||||
|
||||
public void removeFinishedTask(String operationId) {
|
||||
if (!StringUtils.hasLength(operationId)) {
|
||||
return;
|
||||
}
|
||||
FINISHED_TASK_MAP.remove(operationId);
|
||||
}
|
||||
|
||||
public Map<String, FinishedTask> getFinishedTaskMap() {
|
||||
|
||||
return finishedTaskMap;
|
||||
return FINISHED_TASK_MAP;
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -12,6 +12,8 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.constant;
|
||||
|
||||
import lombok.Getter;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
|
@ -19,6 +21,7 @@ import java.util.List;
|
|||
* @author NacosSync
|
||||
* @version $Id: ClusterTypeEnum.java, v 0.1 2018-09-25 下午4:38 NacosSync Exp $$
|
||||
*/
|
||||
@Getter
|
||||
public enum ClusterTypeEnum {
|
||||
|
||||
CS("CS", "configserver集群"),
|
||||
|
@ -32,18 +35,16 @@ public enum ClusterTypeEnum {
|
|||
ZK("ZK", "zookeeper集群");
|
||||
|
||||
|
||||
private String code;
|
||||
private final String code;
|
||||
|
||||
private String desc;
|
||||
|
||||
ClusterTypeEnum(String code, String desc) {
|
||||
this.code = code;
|
||||
this.desc = desc;
|
||||
}
|
||||
|
||||
public static List<String> getClusterTypeCodes() {
|
||||
|
||||
List<String> list = new ArrayList<String>();
|
||||
List<String> list = new ArrayList<>();
|
||||
|
||||
for (ClusterTypeEnum clusterTypeEnum : ClusterTypeEnum.values()) {
|
||||
list.add(clusterTypeEnum.getCode());
|
||||
|
@ -51,42 +52,7 @@ public enum ClusterTypeEnum {
|
|||
return list;
|
||||
}
|
||||
|
||||
/**
|
||||
* Getter method for property <tt>code</tt>.
|
||||
*
|
||||
* @return property value of code
|
||||
*/
|
||||
public String getCode() {
|
||||
return code;
|
||||
}
|
||||
|
||||
/**
|
||||
* Setter method for property <tt>code </tt>.
|
||||
*
|
||||
* @param code value to be assigned to property code
|
||||
*/
|
||||
public void setCode(String code) {
|
||||
this.code = code;
|
||||
}
|
||||
|
||||
/**
|
||||
* Getter method for property <tt>desc</tt>.
|
||||
*
|
||||
* @return property value of desc
|
||||
*/
|
||||
public String getDesc() {
|
||||
return desc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Setter method for property <tt>desc </tt>.
|
||||
*
|
||||
* @param desc value to be assigned to property desc
|
||||
*/
|
||||
public void setDesc(String desc) {
|
||||
this.desc = desc;
|
||||
}
|
||||
|
||||
|
||||
public static boolean contains(String clusterType) {
|
||||
|
||||
for (ClusterTypeEnum clusterTypeEnum : ClusterTypeEnum.values()) {
|
||||
|
|
|
@ -3,10 +3,13 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.constant;
|
||||
|
||||
import lombok.Getter;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: MetricsStatisticsType.java, v 0.1 2019年02月28日 下午2:17 NacosSync Exp $
|
||||
*/
|
||||
@Getter
|
||||
public enum MetricsStatisticsType {
|
||||
|
||||
CACHE_SIZE("nacosSync.finished.taskMap.cacheSize", "任务执行完成缓存列表数"),
|
||||
|
@ -28,15 +31,10 @@ public enum MetricsStatisticsType {
|
|||
/**
|
||||
* metricsName
|
||||
*/
|
||||
private String metricsName;
|
||||
private String desc;
|
||||
private final String metricsName;
|
||||
|
||||
MetricsStatisticsType(String code, String desc) {
|
||||
this.metricsName = code;
|
||||
this.desc = desc;
|
||||
}
|
||||
|
||||
public String getMetricsName() {
|
||||
return metricsName;
|
||||
}
|
||||
|
||||
}
|
|
@ -16,46 +16,23 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.constant;
|
||||
|
||||
import lombok.Getter;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: ResultCodeEnum.java, v 0.1 2018-09-25 PM4:38 NacosSync Exp $$
|
||||
*/
|
||||
@Getter
|
||||
public enum ResultCodeEnum {
|
||||
|
||||
SUCCESS("SUCCESS", "请求成功", "请求成功"),
|
||||
SYSTEM_ERROR("SYSTEM_ERROR", "系统异常", "系统异常");
|
||||
|
||||
private String code;
|
||||
private String errorMessage;
|
||||
private String detail;
|
||||
|
||||
private final String code;
|
||||
|
||||
ResultCodeEnum(String code, String errorMessage, String detail) {
|
||||
this.code = code;
|
||||
this.errorMessage = errorMessage;
|
||||
this.detail = detail;
|
||||
}
|
||||
|
||||
public String getCode() {
|
||||
return code;
|
||||
}
|
||||
|
||||
public void setCode(String code) {
|
||||
this.code = code;
|
||||
}
|
||||
|
||||
public String getErrorMessage() {
|
||||
return errorMessage;
|
||||
}
|
||||
|
||||
public void setErrorMessage(String errorMessage) {
|
||||
this.errorMessage = errorMessage;
|
||||
}
|
||||
|
||||
public String getDetail() {
|
||||
return detail;
|
||||
}
|
||||
|
||||
public void setDetail(String detail) {
|
||||
this.detail = detail;
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -24,11 +24,18 @@ public class SkyWalkerConstants {
|
|||
|
||||
public static final String UNDERLINE = "_";
|
||||
|
||||
public static final String DEST_CLUSTERID_KEY = "destClusterId";
|
||||
public static final String DEST_CLUSTER_ID_KEY = "destClusterId";
|
||||
public static final String GROUP_NAME = "groupName";
|
||||
public static final String SYNC_SOURCE_KEY = "syncSource";
|
||||
public static final String SOURCE_CLUSTERID_KEY = "sourceClusterId";
|
||||
public static final String SOURCE_CLUSTER_ID_KEY = "sourceClusterId";
|
||||
public static final String MANAGEMENT_PORT_KEY="management.port";
|
||||
public static final String MANAGEMENT_CONTEXT_PATH_KEY="management.context-path";
|
||||
|
||||
public static final String SERVICE_NAME_PARAM="serviceNameParam";
|
||||
public static final String GROUP_NAME_PARAM="groupNameParam";
|
||||
public static final String PAGE_NO="pageNo";
|
||||
public static final String PAGE_SIZE="pageSize";
|
||||
public static final String SYNC_INSTANCE_TAG="sync.instance.tag";
|
||||
public static final String NACOS_ALL_SERVICE_NAME = "ALL";
|
||||
|
||||
}
|
||||
|
|
|
@ -16,6 +16,8 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.constant;
|
||||
|
||||
import lombok.Getter;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: TaskStatusEnum.java, v 0.1 2018-09-26 上午2:38 NacosSync Exp $$
|
||||
|
@ -30,51 +32,18 @@ public enum TaskStatusEnum {
|
|||
* delete the task
|
||||
*/
|
||||
DELETE("DELETE", "任务需要被删除");
|
||||
|
||||
|
||||
private String code;
|
||||
private String desc;
|
||||
@Getter
|
||||
private final String code;
|
||||
private final String desc;
|
||||
|
||||
TaskStatusEnum(String code, String desc) {
|
||||
this.code = code;
|
||||
this.desc = desc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Getter method for property <tt>code</tt>.
|
||||
*
|
||||
* @return property value of code
|
||||
*/
|
||||
public String getCode() {
|
||||
return code;
|
||||
}
|
||||
|
||||
/**
|
||||
* Setter method for property <tt>code </tt>.
|
||||
*
|
||||
* @param code value to be assigned to property code
|
||||
*/
|
||||
public void setCode(String code) {
|
||||
this.code = code;
|
||||
}
|
||||
|
||||
/**
|
||||
* Getter method for property <tt>desc</tt>.
|
||||
*
|
||||
* @return property value of desc
|
||||
*/
|
||||
public String getDesc() {
|
||||
return desc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Setter method for property <tt>desc </tt>.
|
||||
*
|
||||
* @param desc value to be assigned to property desc
|
||||
*/
|
||||
public void setDesc(String desc) {
|
||||
this.desc = desc;
|
||||
}
|
||||
|
||||
|
||||
|
||||
public static boolean contains(String code) {
|
||||
|
||||
for (TaskStatusEnum taskStatusEnum : TaskStatusEnum.values()) {
|
||||
|
|
|
@ -16,10 +16,10 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.dao;
|
||||
|
||||
import com.alibaba.nacossync.dao.repository.ClusterRepository;
|
||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.data.domain.Page;
|
||||
import org.springframework.data.domain.PageRequest;
|
||||
import org.springframework.data.domain.Pageable;
|
||||
|
@ -27,9 +27,6 @@ import org.springframework.data.domain.Sort;
|
|||
import org.springframework.data.jpa.domain.Specification;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.dao.repository.ClusterRepository;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
|
||||
import javax.persistence.criteria.CriteriaBuilder;
|
||||
import javax.persistence.criteria.Predicate;
|
||||
import javax.persistence.criteria.Root;
|
||||
|
@ -44,9 +41,12 @@ import java.util.List;
|
|||
@Slf4j
|
||||
public class ClusterAccessService implements PageQueryService<ClusterDO> {
|
||||
|
||||
@Autowired
|
||||
private ClusterRepository clusterRepository;
|
||||
|
||||
private final ClusterRepository clusterRepository;
|
||||
|
||||
public ClusterAccessService(ClusterRepository clusterRepository) {
|
||||
this.clusterRepository = clusterRepository;
|
||||
}
|
||||
|
||||
public ClusterDO insert(ClusterDO clusterDO) {
|
||||
|
||||
return clusterRepository.save(clusterDO);
|
||||
|
@ -103,4 +103,12 @@ public class ClusterAccessService implements PageQueryService<ClusterDO> {
|
|||
predicates.add(criteriaBuilder.like(root.get("clusterName"), "%" + queryCondition.getServiceName() + "%"));
|
||||
return predicates;
|
||||
}
|
||||
|
||||
public int findClusterLevel(String sourceClusterId){
|
||||
ClusterDO clusterDO = clusterRepository.findByClusterId(sourceClusterId);
|
||||
if (clusterDO != null) {
|
||||
return clusterDO.getClusterLevel();
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,10 +14,13 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.dao;
|
||||
|
||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||
import com.alibaba.nacossync.dao.repository.TaskRepository;
|
||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import org.springframework.data.domain.Page;
|
||||
import org.springframework.data.domain.PageRequest;
|
||||
import org.springframework.data.domain.Pageable;
|
||||
|
@ -25,9 +28,6 @@ import org.springframework.data.domain.Sort;
|
|||
import org.springframework.data.jpa.domain.Specification;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.dao.repository.TaskRepository;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
|
||||
import javax.persistence.criteria.CriteriaBuilder;
|
||||
import javax.persistence.criteria.Predicate;
|
||||
import javax.persistence.criteria.Root;
|
||||
|
@ -40,79 +40,94 @@ import java.util.List;
|
|||
*/
|
||||
@Service
|
||||
public class TaskAccessService implements PageQueryService<TaskDO> {
|
||||
|
||||
@Autowired
|
||||
private TaskRepository taskRepository;
|
||||
|
||||
|
||||
private final TaskRepository taskRepository;
|
||||
|
||||
public TaskAccessService(TaskRepository taskRepository) {
|
||||
this.taskRepository = taskRepository;
|
||||
}
|
||||
|
||||
public TaskDO findByTaskId(String taskId) {
|
||||
|
||||
|
||||
return taskRepository.findByTaskId(taskId);
|
||||
}
|
||||
|
||||
|
||||
public void deleteTaskById(String taskId) {
|
||||
taskRepository.deleteByTaskId(taskId);
|
||||
}
|
||||
|
||||
/**
|
||||
* batch delete tasks by taskIds
|
||||
* @author yongchao9
|
||||
*
|
||||
* @param taskIds
|
||||
* @author yongchao9
|
||||
*/
|
||||
public void deleteTaskInBatch(List<String> taskIds) {
|
||||
List<TaskDO> tds=taskRepository.findAllByTaskIdIn(taskIds);
|
||||
taskRepository.deleteInBatch(tds);
|
||||
List<TaskDO> tds = taskRepository.findAllByTaskIdIn(taskIds);
|
||||
taskRepository.deleteAllInBatch(tds);
|
||||
}
|
||||
|
||||
|
||||
public Iterable<TaskDO> findAll() {
|
||||
|
||||
|
||||
return taskRepository.findAll();
|
||||
}
|
||||
|
||||
|
||||
public void addTask(TaskDO taskDO) {
|
||||
|
||||
|
||||
taskRepository.save(taskDO);
|
||||
|
||||
|
||||
}
|
||||
|
||||
|
||||
public int countByDestClusterIdOrSourceClusterId(String destClusterId, String sourceClusterId) {
|
||||
return taskRepository.countByDestClusterIdOrSourceClusterId(destClusterId, sourceClusterId);
|
||||
}
|
||||
|
||||
private Predicate getPredicate(CriteriaBuilder criteriaBuilder, List<Predicate> predicates) {
|
||||
Predicate[] p = new Predicate[predicates.size()];
|
||||
return criteriaBuilder.and(predicates.toArray(p));
|
||||
}
|
||||
|
||||
private List<Predicate> getPredicates(Root<TaskDO> root, CriteriaBuilder criteriaBuilder, QueryCondition queryCondition) {
|
||||
|
||||
|
||||
private List<Predicate> getPredicates(Root<TaskDO> root, CriteriaBuilder criteriaBuilder,
|
||||
QueryCondition queryCondition) {
|
||||
|
||||
List<Predicate> predicates = new ArrayList<>();
|
||||
predicates.add(criteriaBuilder.like(root.get("serviceName"), "%" + queryCondition.getServiceName() + "%"));
|
||||
|
||||
|
||||
return predicates;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public Page<TaskDO> findPageNoCriteria(Integer pageNum, Integer size) {
|
||||
|
||||
|
||||
Pageable pageable = PageRequest.of(pageNum, size, Sort.Direction.DESC, "id");
|
||||
|
||||
|
||||
return taskRepository.findAll(pageable);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public Page<TaskDO> findPageCriteria(Integer pageNum, Integer size, QueryCondition queryCondition) {
|
||||
|
||||
|
||||
Pageable pageable = PageRequest.of(pageNum, size, Sort.Direction.DESC, "id");
|
||||
|
||||
|
||||
return getTaskDOS(queryCondition, pageable);
|
||||
}
|
||||
|
||||
|
||||
private Page<TaskDO> getTaskDOS(QueryCondition queryCondition, Pageable pageable) {
|
||||
return taskRepository.findAll(
|
||||
(Specification<TaskDO>) (root, criteriaQuery, criteriaBuilder) -> {
|
||||
|
||||
List<Predicate> predicates = getPredicates(root,
|
||||
criteriaBuilder, queryCondition);
|
||||
|
||||
return getPredicate(criteriaBuilder, predicates);
|
||||
|
||||
}, pageable);
|
||||
return taskRepository.findAll((Specification<TaskDO>) (root, criteriaQuery, criteriaBuilder) -> {
|
||||
|
||||
List<Predicate> predicates = getPredicates(root, criteriaBuilder, queryCondition);
|
||||
|
||||
return getPredicate(criteriaBuilder, predicates);
|
||||
|
||||
}, pageable);
|
||||
}
|
||||
|
||||
|
||||
public List<TaskDO> findAllByServiceNameEqualAll() {
|
||||
return taskRepository.findAllByServiceNameEqualsIgnoreCase(SkyWalkerConstants.NACOS_ALL_SERVICE_NAME);
|
||||
}
|
||||
|
||||
public List<TaskDO> findAllByServiceNameNotEqualAll() {
|
||||
return taskRepository.findAllByServiceNameNotIgnoreCase(SkyWalkerConstants.NACOS_ALL_SERVICE_NAME);
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
@ -16,13 +16,13 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.dao.repository;
|
||||
|
||||
import javax.transaction.Transactional;
|
||||
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import org.springframework.data.jpa.repository.JpaRepository;
|
||||
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
|
||||
import org.springframework.data.repository.CrudRepository;
|
||||
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import javax.transaction.Transactional;
|
||||
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
|
@ -34,6 +34,6 @@ public interface ClusterRepository extends CrudRepository<ClusterDO, Integer>, J
|
|||
ClusterDO findByClusterId(String clusterId);
|
||||
|
||||
@Transactional
|
||||
int deleteByClusterId(String clusterId);
|
||||
void deleteByClusterId(String clusterId);
|
||||
|
||||
}
|
||||
|
|
|
@ -16,15 +16,13 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.dao.repository;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import javax.transaction.Transactional;
|
||||
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import org.springframework.data.jpa.repository.JpaRepository;
|
||||
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
|
||||
import org.springframework.data.repository.CrudRepository;
|
||||
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import javax.transaction.Transactional;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
|
@ -36,10 +34,15 @@ public interface TaskRepository extends CrudRepository<TaskDO, Integer>, JpaRepo
|
|||
TaskDO findByTaskId(String taskId);
|
||||
|
||||
@Transactional
|
||||
int deleteByTaskId(String taskId);
|
||||
void deleteByTaskId(String taskId);
|
||||
|
||||
List<TaskDO> findAllByTaskIdIn(List<String> taskIds);
|
||||
/**
|
||||
* query service is all,use ns leven sync data
|
||||
*/
|
||||
List<TaskDO> findAllByServiceNameEqualsIgnoreCase(String serviceName);
|
||||
List<TaskDO> findAllByServiceNameNotIgnoreCase(String serviceName);
|
||||
|
||||
List<TaskDO> getAllByWorkerIp(String workerIp);
|
||||
int countByDestClusterIdOrSourceClusterId(String destClusterId,String sourceClusterId);
|
||||
|
||||
}
|
||||
|
|
|
@ -0,0 +1,13 @@
|
|||
package com.alibaba.nacossync.event;
|
||||
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import lombok.Data;
|
||||
|
||||
@Data
|
||||
public class DeleteAllSubTaskEvent {
|
||||
public DeleteAllSubTaskEvent(TaskDO taskDO) {
|
||||
this.taskDO = taskDO;
|
||||
}
|
||||
|
||||
private final TaskDO taskDO;
|
||||
}
|
|
@ -14,23 +14,22 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.event.listener;
|
||||
|
||||
import javax.annotation.PostConstruct;
|
||||
|
||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||
import com.alibaba.nacossync.event.SyncTaskEvent;
|
||||
import com.alibaba.nacossync.extension.SyncManagerService;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.google.common.base.Stopwatch;
|
||||
import com.google.common.eventbus.EventBus;
|
||||
import com.google.common.eventbus.Subscribe;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import javax.annotation.PostConstruct;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
|
@ -39,50 +38,59 @@ import com.google.common.eventbus.Subscribe;
|
|||
@Slf4j
|
||||
@Service
|
||||
public class EventListener {
|
||||
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
|
||||
@Autowired
|
||||
private SyncManagerService syncManagerService;
|
||||
|
||||
@Autowired
|
||||
private EventBus eventBus;
|
||||
|
||||
@Autowired
|
||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
private final SyncManagerService syncManagerService;
|
||||
|
||||
private final EventBus eventBus;
|
||||
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
public EventListener(MetricsManager metricsManager, SyncManagerService syncManagerService, EventBus eventBus,
|
||||
SkyWalkerCacheServices skyWalkerCacheServices) {
|
||||
this.metricsManager = metricsManager;
|
||||
this.syncManagerService = syncManagerService;
|
||||
this.eventBus = eventBus;
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
}
|
||||
|
||||
@PostConstruct
|
||||
public void register() {
|
||||
eventBus.register(this);
|
||||
}
|
||||
|
||||
|
||||
@Subscribe
|
||||
public void listenerSyncTaskEvent(SyncTaskEvent syncTaskEvent) {
|
||||
|
||||
public void sync(SyncTaskEvent syncTaskEvent) {
|
||||
|
||||
try {
|
||||
long start = System.currentTimeMillis();
|
||||
syncManagerService.sync(syncTaskEvent.getTaskDO());
|
||||
skyWalkerCacheServices.addFinishedTask(syncTaskEvent.getTaskDO());
|
||||
metricsManager.record(MetricsStatisticsType.SYNC_TASK_RT, System.currentTimeMillis() - start);
|
||||
|
||||
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||
if (syncManagerService.sync(syncTaskEvent.getTaskDO(), null)) {
|
||||
skyWalkerCacheServices.addFinishedTask(syncTaskEvent.getTaskDO());
|
||||
metricsManager.record(MetricsStatisticsType.SYNC_TASK_RT, stopwatch.elapsed().toMillis());
|
||||
} else {
|
||||
log.warn("syncTaskEvent process error");
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("listenerSyncTaskEvent process error", e);
|
||||
log.warn("syncTaskEvent process error", e);
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
||||
|
||||
@Subscribe
|
||||
public void listenerDeleteTaskEvent(DeleteTaskEvent deleteTaskEvent) {
|
||||
|
||||
public void delete(DeleteTaskEvent deleteTaskEvent) {
|
||||
|
||||
try {
|
||||
long start = System.currentTimeMillis();
|
||||
syncManagerService.delete(deleteTaskEvent.getTaskDO());
|
||||
skyWalkerCacheServices.addFinishedTask(deleteTaskEvent.getTaskDO());
|
||||
metricsManager.record(MetricsStatisticsType.DELETE_TASK_RT, System.currentTimeMillis() - start);
|
||||
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||
if (syncManagerService.delete(deleteTaskEvent.getTaskDO())) {
|
||||
skyWalkerCacheServices.removeFinishedTask(deleteTaskEvent.getTaskDO().getOperationId());
|
||||
metricsManager.record(MetricsStatisticsType.DELETE_TASK_RT, stopwatch.elapsed().toMillis());
|
||||
} else {
|
||||
log.warn("deleteTaskEvent delete failure");
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("listenerDeleteTaskEvent process error", e);
|
||||
log.warn("deleteTaskEvent delete failure.", e);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
@ -12,14 +12,12 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.extension;
|
||||
|
||||
import static com.alibaba.nacossync.util.SkyWalkerUtil.generateSyncKey;
|
||||
|
||||
import com.alibaba.nacos.api.exception.NacosException;
|
||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import com.alibaba.nacossync.util.StringUtils;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.BeansException;
|
||||
import org.springframework.beans.factory.InitializingBean;
|
||||
|
@ -27,6 +25,10 @@ import org.springframework.context.ApplicationContext;
|
|||
import org.springframework.context.ApplicationContextAware;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
import static com.alibaba.nacossync.util.SkyWalkerUtil.generateSyncKey;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: SyncManagerService.java, v 0.1 2018-09-25 PM5:17 NacosSync Exp $$
|
||||
|
@ -37,7 +39,7 @@ public class SyncManagerService implements InitializingBean, ApplicationContextA
|
|||
|
||||
protected final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
private ConcurrentHashMap<String, SyncService> syncServiceMap = new ConcurrentHashMap<String, SyncService>();
|
||||
private final ConcurrentHashMap<String, SyncService> syncServiceMap = new ConcurrentHashMap<String, SyncService>();
|
||||
|
||||
private ApplicationContext applicationContext;
|
||||
|
||||
|
@ -52,19 +54,19 @@ public class SyncManagerService implements InitializingBean, ApplicationContextA
|
|||
|
||||
}
|
||||
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
public boolean sync(TaskDO taskDO, Integer index) {
|
||||
|
||||
return getSyncService(taskDO.getSourceClusterId(), taskDO.getDestClusterId()).sync(taskDO);
|
||||
return getSyncService(taskDO.getSourceClusterId(), taskDO.getDestClusterId()).sync(taskDO, index);
|
||||
|
||||
}
|
||||
|
||||
@Override
|
||||
public void afterPropertiesSet() {
|
||||
this.applicationContext.getBeansOfType(SyncService.class).forEach((key, value) -> {
|
||||
this.applicationContext.getBeansWithAnnotation(NacosSyncService.class).forEach((key, value) -> {
|
||||
NacosSyncService nacosSyncService = value.getClass().getAnnotation(NacosSyncService.class);
|
||||
ClusterTypeEnum sourceCluster = nacosSyncService.sourceCluster();
|
||||
ClusterTypeEnum destinationCluster = nacosSyncService.destinationCluster();
|
||||
syncServiceMap.put(generateSyncKey(sourceCluster, destinationCluster), value);
|
||||
syncServiceMap.put(generateSyncKey(sourceCluster, destinationCluster), (SyncService) value);
|
||||
});
|
||||
}
|
||||
|
||||
|
@ -74,7 +76,9 @@ public class SyncManagerService implements InitializingBean, ApplicationContextA
|
|||
}
|
||||
|
||||
public SyncService getSyncService(String sourceClusterId, String destClusterId) {
|
||||
|
||||
if (StringUtils.isEmpty(sourceClusterId) || StringUtils.isEmpty(destClusterId)) {
|
||||
throw new IllegalArgumentException("Source cluster id and destination cluster id must not be null or empty");
|
||||
}
|
||||
ClusterTypeEnum sourceClusterType = this.skyWalkerCacheServices.getClusterType(sourceClusterId);
|
||||
ClusterTypeEnum destClusterType = this.skyWalkerCacheServices.getClusterType(destClusterId);
|
||||
|
||||
|
|
|
@ -14,9 +14,10 @@ package com.alibaba.nacossync.extension;
|
|||
|
||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import java.util.Map;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: SyncManagerService.java, v 0.1 2018-09-25 下午5:17 NacosSync Exp $$
|
||||
|
@ -30,20 +31,22 @@ public interface SyncService {
|
|||
* @return
|
||||
*/
|
||||
boolean delete(TaskDO taskDO);
|
||||
|
||||
|
||||
/**
|
||||
* execute sync
|
||||
*
|
||||
* @param taskDO
|
||||
* @param index
|
||||
* @return
|
||||
*/
|
||||
boolean sync(TaskDO taskDO);
|
||||
boolean sync(TaskDO taskDO, Integer index);
|
||||
|
||||
/**
|
||||
* Determines that the current instance data is from another source cluster
|
||||
*/
|
||||
default boolean needSync(Map<String, String> sourceMetaData) {
|
||||
return StringUtils.isBlank(sourceMetaData.get(SkyWalkerConstants.SOURCE_CLUSTERID_KEY));
|
||||
boolean syncTag = StringUtils.isBlank(sourceMetaData.get(SkyWalkerConstants.SYNC_INSTANCE_TAG));
|
||||
boolean blank = StringUtils.isBlank(sourceMetaData.get(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY));
|
||||
return syncTag && blank;
|
||||
}
|
||||
|
||||
/**
|
||||
|
@ -51,7 +54,7 @@ public interface SyncService {
|
|||
* cluster ID of the task
|
||||
*/
|
||||
default boolean needDelete(Map<String, String> destMetaData, TaskDO taskDO) {
|
||||
return StringUtils.equals(destMetaData.get(SkyWalkerConstants.SOURCE_CLUSTERID_KEY),
|
||||
return StringUtils.equals(destMetaData.get(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY),
|
||||
taskDO.getSourceClusterId());
|
||||
}
|
||||
|
||||
|
|
|
@ -1,15 +0,0 @@
|
|||
package com.alibaba.nacossync.extension.client;
|
||||
|
||||
import lombok.Data;
|
||||
|
||||
@Data
|
||||
public class InstanceQueryModel {
|
||||
|
||||
private String sourceClusterId;
|
||||
private String destClusterId;
|
||||
private String serviceName;
|
||||
private String groupName;
|
||||
private String version;
|
||||
private int pageNo;
|
||||
private int pageSize;
|
||||
}
|
|
@ -1,11 +0,0 @@
|
|||
package com.alibaba.nacossync.extension.client;
|
||||
|
||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
||||
import java.util.List;
|
||||
|
||||
public interface SyncQueryClient {
|
||||
|
||||
|
||||
List<TaskModel> getAllInstance(InstanceQueryModel instanceQueryModel);
|
||||
|
||||
}
|
|
@ -1,54 +0,0 @@
|
|||
package com.alibaba.nacossync.extension.client.impl;
|
||||
|
||||
import com.alibaba.nacos.api.exception.NacosException;
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
import com.alibaba.nacos.api.naming.pojo.ListView;
|
||||
import com.alibaba.nacossync.extension.client.InstanceQueryModel;
|
||||
import com.alibaba.nacossync.extension.client.SyncQueryClient;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.stream.Collectors;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
@Service
|
||||
@Slf4j
|
||||
public class NacosSyncQueryClientImpl implements SyncQueryClient {
|
||||
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
|
||||
public NacosSyncQueryClientImpl(
|
||||
NacosServerHolder nacosServerHolder) {
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<TaskModel> getAllInstance(InstanceQueryModel instanceQueryModel) {
|
||||
NamingService namingService = nacosServerHolder
|
||||
.get(instanceQueryModel.getSourceClusterId(), instanceQueryModel.getGroupName());
|
||||
try {
|
||||
ListView<String> servicesOfServer = namingService
|
||||
.getServicesOfServer(instanceQueryModel.getPageNo(),
|
||||
instanceQueryModel.getPageSize());
|
||||
return servicesOfServer.getData().stream()
|
||||
.map(serviceName -> buildTaskModel(instanceQueryModel, serviceName))
|
||||
.collect(Collectors.toList());
|
||||
|
||||
} catch (NacosException e) {
|
||||
log.error("When using nacos client failure query tasks", e);
|
||||
return Collections.emptyList();
|
||||
}
|
||||
}
|
||||
|
||||
private TaskModel buildTaskModel(InstanceQueryModel instanceQueryModel, String serviceName) {
|
||||
TaskModel taskModel = new TaskModel();
|
||||
taskModel.setServiceName(serviceName);
|
||||
taskModel.setSourceClusterId(instanceQueryModel.getSourceClusterId());
|
||||
taskModel.setDestClusterId(instanceQueryModel.getDestClusterId());
|
||||
return taskModel;
|
||||
}
|
||||
|
||||
|
||||
}
|
|
@ -29,11 +29,11 @@ import java.util.concurrent.TimeUnit;
|
|||
*/
|
||||
@Slf4j
|
||||
public class EurekaBeatReactor {
|
||||
private ScheduledExecutorService executorService;
|
||||
private final ScheduledExecutorService executorService;
|
||||
|
||||
private volatile long clientBeatInterval = 5 * 1000;
|
||||
private static final long CLIENT_BEAT_INTERVAL = 5 * 1000;
|
||||
private final Map<String, InstanceInfo> eurekaBeat = new ConcurrentHashMap<>();
|
||||
private EurekaHttpClient eurekaHttpClient;
|
||||
private final EurekaHttpClient eurekaHttpClient;
|
||||
|
||||
public EurekaBeatReactor(EurekaHttpClient eurekaHttpClient) {
|
||||
this.eurekaHttpClient = eurekaHttpClient;
|
||||
|
@ -71,7 +71,7 @@ public class EurekaBeatReactor {
|
|||
} catch (Exception e) {
|
||||
log.error("[CLIENT-BEAT] Exception while scheduling beat.", e);
|
||||
} finally {
|
||||
executorService.schedule(this, clientBeatInterval, TimeUnit.MILLISECONDS);
|
||||
executorService.schedule(this, CLIENT_BEAT_INTERVAL, TimeUnit.MILLISECONDS);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
@ -26,8 +26,8 @@ import java.util.Objects;
|
|||
* @date 2019-06-26
|
||||
*/
|
||||
public class EurekaNamingService {
|
||||
private EurekaHttpClient eurekaHttpClient;
|
||||
private EurekaBeatReactor beatReactor;
|
||||
private final EurekaHttpClient eurekaHttpClient;
|
||||
private final EurekaBeatReactor beatReactor;
|
||||
|
||||
|
||||
public EurekaNamingService(EurekaHttpClient eurekaHttpClient) {
|
||||
|
|
|
@ -25,7 +25,7 @@ import java.util.function.Consumer;
|
|||
*/
|
||||
@Service
|
||||
public class SpecialSyncEventBus {
|
||||
private ConcurrentHashMap<String, SpecialSyncEvent> specialSyncEventRegistry = new ConcurrentHashMap<>();
|
||||
private final ConcurrentHashMap<String, SpecialSyncEvent> specialSyncEventRegistry = new ConcurrentHashMap<>();
|
||||
|
||||
public void subscribe(TaskDO taskDO, Consumer<TaskDO> syncAction) {
|
||||
SpecialSyncEvent specialSyncEvent = new SpecialSyncEvent();
|
||||
|
|
|
@ -21,7 +21,6 @@ import com.alibaba.nacossync.pojo.model.TaskDO;
|
|||
import com.google.common.eventbus.EventBus;
|
||||
import com.google.common.eventbus.Subscribe;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import javax.annotation.PostConstruct;
|
||||
|
@ -34,9 +33,12 @@ import java.util.function.Consumer;
|
|||
@Service
|
||||
@Slf4j
|
||||
public class SpecialSyncEventListener {
|
||||
@Autowired
|
||||
private EventBus eventBus;
|
||||
|
||||
private final EventBus eventBus;
|
||||
|
||||
public SpecialSyncEventListener(EventBus eventBus) {
|
||||
this.eventBus = eventBus;
|
||||
}
|
||||
|
||||
@PostConstruct
|
||||
public void init() {
|
||||
eventBus.register(this);
|
||||
|
|
|
@ -13,13 +13,10 @@
|
|||
package com.alibaba.nacossync.extension.holder;
|
||||
|
||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||
import com.google.common.base.Joiner;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.function.Supplier;
|
||||
|
||||
|
@ -28,40 +25,36 @@ import java.util.function.Supplier;
|
|||
* @date 2018-12-24 22:08
|
||||
*/
|
||||
@Slf4j
|
||||
public abstract class AbstractServerHolderImpl<T> implements Holder {
|
||||
|
||||
public abstract class AbstractServerHolderImpl<T> implements Holder<T> {
|
||||
|
||||
private final Map<String, T> serviceMap = new ConcurrentHashMap<>();
|
||||
|
||||
|
||||
@Autowired
|
||||
protected SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
@Override
|
||||
public T get(String clusterId, String namespace) {
|
||||
final String finalNamespace = Optional.ofNullable(namespace).orElse(Strings.EMPTY);
|
||||
String key = Joiner.on("_").join(clusterId, finalNamespace);
|
||||
public T get(String clusterId) {
|
||||
|
||||
serviceMap.computeIfAbsent(key, clusterKey -> {
|
||||
return serviceMap.computeIfAbsent(clusterId, clusterKey -> {
|
||||
try {
|
||||
log.info("Starting create cluster server, clusterId={}", clusterId);
|
||||
return createServer(clusterId, () -> skyWalkerCacheServices.getClusterConnectKey(clusterId),
|
||||
finalNamespace);
|
||||
return createServer(clusterId, () -> skyWalkerCacheServices.getClusterConnectKey(clusterId));
|
||||
} catch (Exception e) {
|
||||
log.error(String.format("clusterId=%s, start server failed", clusterId), e);
|
||||
return null;
|
||||
}
|
||||
});
|
||||
|
||||
return serviceMap.get(key);
|
||||
}
|
||||
|
||||
/**
|
||||
* Create real cluster client instance
|
||||
*
|
||||
* @param clusterId cluster id
|
||||
* @param clusterId cluster id
|
||||
* @param serverAddressSupplier server address
|
||||
* @param namespace name space
|
||||
* @return cluster client instance
|
||||
*/
|
||||
abstract T createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace)
|
||||
abstract T createServer(String clusterId, Supplier<String> serverAddressSupplier)
|
||||
throws Exception;
|
||||
|
||||
}
|
||||
|
|
|
@ -30,7 +30,7 @@ public class ConsulServerHolder extends AbstractServerHolderImpl<ConsulClient> {
|
|||
public static final String HTTP = "http://";
|
||||
|
||||
@Override
|
||||
ConsulClient createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace) throws Exception {
|
||||
ConsulClient createServer(String clusterId, Supplier<String> serverAddressSupplier) throws Exception {
|
||||
String serverAddress = serverAddressSupplier.get();
|
||||
serverAddress = serverAddress.startsWith(HTTP) ? serverAddress : HTTP + serverAddress;
|
||||
URL url = new URL(serverAddress);
|
||||
|
|
|
@ -10,6 +10,7 @@
|
|||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.extension.holder;
|
||||
|
||||
import com.alibaba.nacossync.extension.eureka.EurekaNamingService;
|
||||
|
@ -30,12 +31,27 @@ import java.util.function.Supplier;
|
|||
@Service
|
||||
@Slf4j
|
||||
public class EurekaServerHolder extends AbstractServerHolderImpl<EurekaNamingService> {
|
||||
|
||||
private static final String HTTP_PREFIX = "http://";
|
||||
|
||||
private static final String HTTPS_PREFIX = "https://";
|
||||
|
||||
@Override
|
||||
EurekaNamingService createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace) throws Exception {
|
||||
RestTemplateTransportClientFactory restTemplateTransportClientFactory =
|
||||
new RestTemplateTransportClientFactory();
|
||||
EurekaEndpoint eurekaEndpoint = new DefaultEndpoint(serverAddressSupplier.get());
|
||||
EurekaNamingService createServer(String clusterId, Supplier<String> serverAddressSupplier) throws Exception {
|
||||
RestTemplateTransportClientFactory restTemplateTransportClientFactory = new RestTemplateTransportClientFactory();
|
||||
EurekaEndpoint eurekaEndpoint = new DefaultEndpoint(addHttpPrefix(serverAddressSupplier.get()));
|
||||
EurekaHttpClient eurekaHttpClient = restTemplateTransportClientFactory.newClient(eurekaEndpoint);
|
||||
return new EurekaNamingService(eurekaHttpClient);
|
||||
}
|
||||
|
||||
public String addHttpPrefix(String input) {
|
||||
if (input == null || input.isEmpty()) {
|
||||
return input;
|
||||
}
|
||||
if (!input.startsWith(HTTP_PREFIX) && !input.startsWith(HTTPS_PREFIX)) {
|
||||
input = HTTP_PREFIX + input;
|
||||
}
|
||||
|
||||
return input;
|
||||
}
|
||||
}
|
||||
|
|
|
@ -22,9 +22,8 @@ public interface Holder<T> {
|
|||
/**
|
||||
* Through the cluster ID and namespace fetch cluster client service
|
||||
* @param clusterId cluster id
|
||||
* @param namespace name space
|
||||
* @return
|
||||
* @throws Exception
|
||||
*/
|
||||
T get(String clusterId, String namespace) throws Exception;
|
||||
T get(String clusterId) throws Exception;
|
||||
}
|
||||
|
|
|
@ -18,12 +18,14 @@ import com.alibaba.nacos.api.naming.NamingService;
|
|||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import com.google.common.base.Joiner;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
import java.util.Properties;
|
||||
import java.util.function.Supplier;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
/**
|
||||
* @author paderlol
|
||||
|
@ -34,13 +36,13 @@ import org.springframework.stereotype.Service;
|
|||
public class NacosServerHolder extends AbstractServerHolderImpl<NamingService> {
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
|
||||
public NacosServerHolder(ClusterAccessService clusterAccessService) {
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
NamingService createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace)
|
||||
NamingService createServer(String clusterId, Supplier<String> serverAddressSupplier)
|
||||
throws Exception {
|
||||
List<String> allClusterConnectKey = skyWalkerCacheServices
|
||||
.getAllClusterConnectKey(clusterId);
|
||||
|
@ -48,7 +50,8 @@ public class NacosServerHolder extends AbstractServerHolderImpl<NamingService> {
|
|||
String serverList = Joiner.on(",").join(allClusterConnectKey);
|
||||
Properties properties = new Properties();
|
||||
properties.setProperty(PropertyKeyConst.SERVER_ADDR, serverList);
|
||||
properties.setProperty(PropertyKeyConst.NAMESPACE, namespace);
|
||||
properties.setProperty(PropertyKeyConst.NAMESPACE, Optional.ofNullable(clusterDO.getNamespace()).orElse(
|
||||
Strings.EMPTY));
|
||||
Optional.ofNullable(clusterDO.getUserName()).ifPresent(value ->
|
||||
properties.setProperty(PropertyKeyConst.USERNAME, value)
|
||||
);
|
||||
|
@ -58,4 +61,5 @@ public class NacosServerHolder extends AbstractServerHolderImpl<NamingService> {
|
|||
);
|
||||
return NamingFactory.createNamingService(properties);
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
@ -32,7 +32,7 @@ public class ZookeeperServerHolder extends AbstractServerHolderImpl<CuratorFrame
|
|||
|
||||
|
||||
@Override
|
||||
CuratorFramework createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace) {
|
||||
CuratorFramework createServer(String clusterId, Supplier<String> serverAddressSupplier) {
|
||||
List<String> allClusterConnectKey = skyWalkerCacheServices
|
||||
.getAllClusterConnectKey(clusterId);
|
||||
String serverList = Joiner.on(",").join(allClusterConnectKey);
|
||||
|
|
|
@ -0,0 +1,265 @@
|
|||
package com.alibaba.nacossync.extension.impl;
|
||||
|
||||
import com.alibaba.nacos.api.exception.NacosException;
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
import com.alibaba.nacos.api.naming.listener.EventListener;
|
||||
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
||||
import com.alibaba.nacos.api.naming.pojo.Instance;
|
||||
import com.alibaba.nacos.common.utils.CollectionUtils;
|
||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||
import com.alibaba.nacossync.extension.SyncService;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||
import com.google.common.base.Stopwatch;
|
||||
import lombok.Getter;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
import javax.annotation.PostConstruct;
|
||||
import javax.annotation.PreDestroy;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collection;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.TreeSet;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||
|
||||
@Slf4j
|
||||
public abstract class AbstractNacosSync implements SyncService {
|
||||
|
||||
private final Map<String, EventListener> listenerMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final Map<String, Set<String>> sourceInstanceSnapshot = new ConcurrentHashMap<>();
|
||||
|
||||
private final Map<String, Integer> syncTaskTap = new ConcurrentHashMap<>();
|
||||
|
||||
private final ConcurrentHashMap<String, TaskDO> allSyncTaskMap = new ConcurrentHashMap<>();
|
||||
private ScheduledExecutorService executorService;
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
|
||||
@Getter
|
||||
@Autowired
|
||||
private NacosServerHolder nacosServerHolder;
|
||||
|
||||
|
||||
/**
|
||||
* Due to network issues or other reasons, the Nacos Sync synchronization tasks may fail,
|
||||
* resulting in the target cluster's registry missing synchronized instances.
|
||||
* To prevent the target cluster's registry from missing synchronized instances for an extended period,
|
||||
* a fallback worker thread is started every 5 minutes to execute all synchronization tasks.
|
||||
*/
|
||||
@PostConstruct
|
||||
public void afterPropertiesSet() {
|
||||
initializeExecutorService();
|
||||
scheduleSyncTasks();
|
||||
}
|
||||
|
||||
@PreDestroy
|
||||
public void destroy() {
|
||||
if (executorService != null && !executorService.isShutdown()) {
|
||||
executorService.shutdown();
|
||||
try {
|
||||
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||
executorService.shutdownNow();
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
executorService.shutdownNow();
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void initializeExecutorService() {
|
||||
executorService = Executors.newSingleThreadScheduledExecutor(r -> {
|
||||
Thread t = new Thread(r);
|
||||
t.setDaemon(true);
|
||||
t.setName("com.alibaba.nacossync.basic.synctask");
|
||||
return t;
|
||||
});
|
||||
}
|
||||
|
||||
private void scheduleSyncTasks() {
|
||||
executorService.scheduleWithFixedDelay(this::executeSyncTasks, 0, 300, TimeUnit.SECONDS);
|
||||
}
|
||||
|
||||
private void executeSyncTasks() {
|
||||
if (allSyncTaskMap.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
|
||||
Collection<TaskDO> taskCollections = allSyncTaskMap.values();
|
||||
List<TaskDO> taskDOList = new ArrayList<>(taskCollections);
|
||||
|
||||
if (CollectionUtils.isNotEmpty(taskDOList)) {
|
||||
BatchTaskExecutor.batchOperation(taskDOList, this::executeTask);
|
||||
}
|
||||
}
|
||||
|
||||
private void executeTask(TaskDO task) {
|
||||
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||
String taskId = task.getTaskId();
|
||||
try {
|
||||
NamingService sourceNamingService = nacosServerHolder.get(task.getSourceClusterId());
|
||||
doSync(taskId, task, sourceNamingService);
|
||||
} catch (NacosException e) {
|
||||
log.error("sync task from nacos to nacos failed, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
} catch (Exception e) {
|
||||
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
} finally {
|
||||
stopwatch.stop();
|
||||
log.debug("Task execution time for taskId {}: {} ms", taskId, stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean delete(TaskDO taskDO) {
|
||||
String taskId = taskDO.getTaskId();
|
||||
try {
|
||||
|
||||
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||
//移除订阅
|
||||
EventListener listener = listenerMap.remove(taskId);
|
||||
if (listener!= null) {
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()), listener);
|
||||
}
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
listenerMap.remove(taskId));
|
||||
sourceInstanceSnapshot.remove(taskId);
|
||||
allSyncTaskMap.remove(taskId);
|
||||
|
||||
// 删除目标集群中同步的实例列表
|
||||
deregisterInstance(taskDO);
|
||||
}catch (NacosException e) {
|
||||
log.error("Delete task from nacos to specify destination was failed, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
return false;
|
||||
} catch (Exception e) {
|
||||
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO, Integer index) {
|
||||
String taskId = taskDO.getTaskId();
|
||||
try {
|
||||
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||
allSyncTaskMap.put(taskId, taskDO);
|
||||
//防止暂停同步任务后,重新同步/或删除任务以后新建任务不会再接收到新的事件导致不能同步,所以每次订阅事件之前,先全量同步一次任务
|
||||
doSync(taskId, taskDO, sourceNamingService);
|
||||
this.listenerMap.putIfAbsent(taskId, event -> {
|
||||
if (event instanceof NamingEvent) {
|
||||
try {
|
||||
doSync(taskId, taskDO, sourceNamingService);
|
||||
} catch (Exception e) {
|
||||
log.error("event process fail, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
}
|
||||
}
|
||||
});
|
||||
sourceNamingService.subscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
listenerMap.get(taskId));
|
||||
}catch (NacosException e) {
|
||||
log.error("Nacos sync task process fail, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
return false;
|
||||
} catch (Exception e) {
|
||||
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private void doSync(String taskId, TaskDO taskDO, NamingService sourceNamingService) throws Exception {
|
||||
if (syncTaskTap.putIfAbsent(taskId, 1) != null) {
|
||||
log.info("任务Id:{}上一个同步任务尚未结束", taskId);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
// 直接从本地保存的serviceInfoMap中取订阅的服务实例
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||
// 先删除不存在的
|
||||
this.removeInvalidInstance(taskDO, sourceInstances);
|
||||
|
||||
// 同步实例
|
||||
this.syncNewInstance(taskDO, sourceInstances);
|
||||
} finally {
|
||||
syncTaskTap.remove(taskId);
|
||||
}
|
||||
}
|
||||
|
||||
private void syncNewInstance(TaskDO taskDO, List<Instance> sourceInstances) throws NacosException {
|
||||
Set<String> latestSyncInstance = new TreeSet<>();
|
||||
//再次添加新实例
|
||||
String taskId = taskDO.getTaskId();
|
||||
Set<String> instanceKeys = sourceInstanceSnapshot.get(taskId);
|
||||
for (Instance instance : sourceInstances) {
|
||||
if (needSync(instance.getMetadata())) {
|
||||
String instanceKey = composeInstanceKey(instance.getIp(), instance.getPort());
|
||||
if (CollectionUtils.isEmpty(instanceKeys) || !instanceKeys.contains(instanceKey)) {
|
||||
register(taskDO, instance);
|
||||
}
|
||||
latestSyncInstance.add(instanceKey);
|
||||
}
|
||||
}
|
||||
|
||||
if (CollectionUtils.isNotEmpty(latestSyncInstance)) {
|
||||
log.info("任务Id:{},已同步实例个数:{}", taskId, latestSyncInstance.size());
|
||||
sourceInstanceSnapshot.put(taskId, latestSyncInstance);
|
||||
} else {
|
||||
// latestSyncInstance为空表示源集群中需要同步的所有实例(即非nacos-sync同步过来的实例)已经下线,清除本地持有快照
|
||||
sourceInstanceSnapshot.remove(taskId);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
private void removeInvalidInstance(TaskDO taskDO, List<Instance> sourceInstances) throws Exception {
|
||||
String taskId = taskDO.getTaskId();
|
||||
if (this.sourceInstanceSnapshot.containsKey(taskId)) {
|
||||
Set<String> oldInstanceKeys = this.sourceInstanceSnapshot.get(taskId);
|
||||
Set<String> newInstanceKeys = sourceInstances.stream()
|
||||
.map(instance -> composeInstanceKey(instance.getIp(), instance.getPort()))
|
||||
.collect(Collectors.toSet());
|
||||
oldInstanceKeys.removeAll(newInstanceKeys);
|
||||
if (CollectionUtils.isNotEmpty(oldInstanceKeys)) {
|
||||
log.info("任务Id:{},移除无效同步实例:{}", taskId, oldInstanceKeys);
|
||||
removeInvalidInstance(taskDO, oldInstanceKeys);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean needDelete(Map<String, String> destMetaData, TaskDO taskDO) {
|
||||
return SyncService.super.needDelete(destMetaData, taskDO);
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean needSync(Map<String, String> sourceMetaData) {
|
||||
return SyncService.super.needSync(sourceMetaData);
|
||||
}
|
||||
|
||||
public abstract String composeInstanceKey(String ip, int port);
|
||||
|
||||
public abstract void register(TaskDO taskDO, Instance instance);
|
||||
|
||||
public abstract void deregisterInstance(TaskDO taskDO) throws Exception;
|
||||
|
||||
public abstract void removeInvalidInstance(TaskDO taskDO, Set<String> invalidInstanceKeys) throws Exception;
|
||||
|
||||
}
|
|
@ -27,18 +27,22 @@ import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
|||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.ConsulUtils;
|
||||
import com.alibaba.nacossync.util.NacosUtils;
|
||||
import com.ecwid.consul.v1.ConsulClient;
|
||||
import com.ecwid.consul.v1.QueryParams;
|
||||
import com.ecwid.consul.v1.Response;
|
||||
import com.ecwid.consul.v1.health.model.HealthService;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
import java.util.*;
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* Consul 同步 Nacos
|
||||
*
|
||||
*
|
||||
* @author paderlol
|
||||
* @date: 2018-12-31 16:25
|
||||
*/
|
||||
|
@ -46,8 +50,7 @@ import java.util.*;
|
|||
@NacosSyncService(sourceCluster = ClusterTypeEnum.CONSUL, destinationCluster = ClusterTypeEnum.NACOS)
|
||||
public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
private final ConsulServerHolder consulServerHolder;
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
@ -56,14 +59,15 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
|||
|
||||
private final SpecialSyncEventBus specialSyncEventBus;
|
||||
|
||||
@Autowired
|
||||
|
||||
public ConsulSyncToNacosServiceImpl(ConsulServerHolder consulServerHolder,
|
||||
SkyWalkerCacheServices skyWalkerCacheServices, NacosServerHolder nacosServerHolder,
|
||||
SpecialSyncEventBus specialSyncEventBus) {
|
||||
SpecialSyncEventBus specialSyncEventBus, MetricsManager metricsManager) {
|
||||
this.consulServerHolder = consulServerHolder;
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
this.specialSyncEventBus = specialSyncEventBus;
|
||||
this.metricsManager = metricsManager;
|
||||
}
|
||||
|
||||
@Override
|
||||
|
@ -71,12 +75,15 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
|||
|
||||
try {
|
||||
specialSyncEventBus.unsubscribe(taskDO);
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
||||
List<Instance> allInstances = destNamingService.getAllInstances(taskDO.getServiceName());
|
||||
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
List<Instance> allInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()));
|
||||
for (Instance instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -89,17 +96,17 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
|||
}
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
public boolean sync(TaskDO taskDO, Integer index) {
|
||||
try {
|
||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getSourceClusterId(), null);
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getSourceClusterId());
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
Response<List<HealthService>> response =
|
||||
consulClient.getHealthServices(taskDO.getServiceName(), true, QueryParams.DEFAULT);
|
||||
List<HealthService> healthServiceList = response.getValue();
|
||||
Set<String> instanceKeys = new HashSet<>();
|
||||
overrideAllInstance(taskDO, destNamingService, healthServiceList, instanceKeys);
|
||||
cleanAllOldInstance(taskDO, destNamingService, instanceKeys);
|
||||
specialSyncEventBus.subscribe(taskDO, this::sync);
|
||||
specialSyncEventBus.subscribe(taskDO, t->sync(t, index));
|
||||
} catch (Exception e) {
|
||||
log.error("Sync task from consul to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
|
@ -115,7 +122,8 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
|||
if (needDelete(instance.getMetadata(), taskDO)
|
||||
&& !instanceKeys.contains(composeInstanceKey(instance.getIp(), instance.getPort()))) {
|
||||
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -125,6 +133,7 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
|||
for (HealthService healthService : healthServiceList) {
|
||||
if (needSync(ConsulUtils.transferMetadata(healthService.getService().getTags()))) {
|
||||
destNamingService.registerInstance(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
buildSyncInstance(healthService, taskDO));
|
||||
instanceKeys.add(composeInstanceKey(healthService.getService().getAddress(),
|
||||
healthService.getService().getPort()));
|
||||
|
@ -137,10 +146,10 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
|||
temp.setIp(instance.getService().getAddress());
|
||||
temp.setPort(instance.getService().getPort());
|
||||
Map<String, String> metaData = new HashMap<>(ConsulUtils.transferMetadata(instance.getService().getTags()));
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||
temp.setMetadata(metaData);
|
||||
return temp;
|
||||
}
|
||||
|
|
|
@ -27,6 +27,7 @@ import com.alibaba.nacossync.extension.holder.EurekaServerHolder;
|
|||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.NacosUtils;
|
||||
import com.netflix.appinfo.InstanceInfo;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
@ -38,7 +39,7 @@ import java.util.Map;
|
|||
|
||||
/**
|
||||
* eureka
|
||||
*
|
||||
*
|
||||
* @author paderlol
|
||||
* @date: 2018-12-31 16:25
|
||||
*/
|
||||
|
@ -71,8 +72,10 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
|||
|
||||
try {
|
||||
specialSyncEventBus.unsubscribe(taskDO);
|
||||
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId(), null);
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
||||
|
||||
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId());
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
|
||||
List<InstanceInfo> eurekaInstances = eurekaNamingService.getApplications(taskDO.getServiceName());
|
||||
deleteAllInstanceFromEureka(taskDO, destNamingService, eurekaInstances);
|
||||
|
||||
|
@ -85,12 +88,15 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
|||
}
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
public boolean sync(TaskDO taskDO,Integer index) {
|
||||
try {
|
||||
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId(), null);
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
||||
|
||||
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId());
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
|
||||
List<InstanceInfo> eurekaInstances = eurekaNamingService.getApplications(taskDO.getServiceName());
|
||||
List<Instance> nacosInstances = destNamingService.getAllInstances(taskDO.getServiceName());
|
||||
List<Instance> nacosInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()));
|
||||
|
||||
if (CollectionUtils.isEmpty(eurekaInstances)) {
|
||||
// Clear all instance from Nacos
|
||||
|
@ -102,7 +108,7 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
|||
}
|
||||
addValidInstance(taskDO, destNamingService, eurekaInstances);
|
||||
}
|
||||
specialSyncEventBus.subscribe(taskDO, this::sync);
|
||||
specialSyncEventBus.subscribe(taskDO, t->sync(t, index));
|
||||
} catch (Exception e) {
|
||||
log.error("sync task from eureka to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
|
@ -117,18 +123,25 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
|||
if (needSync(instance.getMetadata())) {
|
||||
log.info("Add service instance from Eureka, serviceName={}, Ip={}, port={}",
|
||||
instance.getAppName(), instance.getIPAddr(), instance.getPort());
|
||||
destNamingService.registerInstance(taskDO.getServiceName(), buildSyncInstance(instance, taskDO));
|
||||
destNamingService.registerInstance(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), buildSyncInstance(instance,
|
||||
taskDO));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void deleteAllInstanceFromEureka(TaskDO taskDO, NamingService destNamingService, List<InstanceInfo> eurekaInstances)
|
||||
throws NacosException {
|
||||
|
||||
private void deleteAllInstanceFromEureka(TaskDO taskDO, NamingService destNamingService,
|
||||
List<InstanceInfo> eurekaInstances)
|
||||
throws NacosException {
|
||||
if (CollectionUtils.isEmpty(eurekaInstances)) {
|
||||
return;
|
||||
}
|
||||
for (InstanceInfo instance : eurekaInstances) {
|
||||
if (needSync(instance.getMetadata())) {
|
||||
log.info("Delete service instance from Eureka, serviceName={}, Ip={}, port={}",
|
||||
instance.getAppName(), instance.getIPAddr(), instance.getPort());
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), buildSyncInstance(instance, taskDO));
|
||||
instance.getAppName(), instance.getIPAddr(), instance.getPort());
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), buildSyncInstance(instance, taskDO));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -139,7 +152,8 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
|||
if (!isExistInEurekaInstance(eurekaInstances, instance) && needDelete(instance.getMetadata(), taskDO)) {
|
||||
log.info("Remove invalid service instance from Nacos, serviceName={}, Ip={}, port={}",
|
||||
instance.getServiceName(), instance.getIp(), instance.getPort());
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -154,7 +168,8 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
|||
throws NacosException {
|
||||
for (Instance instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance);
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -168,10 +183,10 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
|||
temp.setHealthy(true);
|
||||
|
||||
Map<String, String> metaData = new HashMap<>(instance.getMetadata());
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||
temp.setMetadata(metaData);
|
||||
return temp;
|
||||
}
|
||||
|
|
|
@ -10,21 +10,15 @@
|
|||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.extension.impl;
|
||||
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
import com.alibaba.nacos.api.naming.listener.EventListener;
|
||||
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
||||
import com.alibaba.nacos.api.naming.pojo.Instance;
|
||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||
import com.alibaba.nacossync.extension.SyncService;
|
||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||
import com.alibaba.nacossync.extension.holder.ConsulServerHolder;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.ConsulUtils;
|
||||
import com.ecwid.consul.v1.ConsulClient;
|
||||
|
@ -33,15 +27,13 @@ import com.ecwid.consul.v1.Response;
|
|||
import com.ecwid.consul.v1.agent.model.NewService;
|
||||
import com.ecwid.consul.v1.health.model.HealthService;
|
||||
import com.google.common.collect.Lists;
|
||||
import java.net.URLEncoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.io.UnsupportedEncodingException;
|
||||
import java.net.URLEncoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
|
@ -49,107 +41,66 @@ import java.util.stream.Collectors;
|
|||
*/
|
||||
@Slf4j
|
||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.CONSUL)
|
||||
public class NacosSyncToConsulServiceImpl implements SyncService {
|
||||
private Map<String, EventListener> nacosListenerMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
public class NacosSyncToConsulServiceImpl extends AbstractNacosSync {
|
||||
|
||||
|
||||
private static final String DELIMITER = "=";
|
||||
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
|
||||
private final ConsulServerHolder consulServerHolder;
|
||||
|
||||
public NacosSyncToConsulServiceImpl(MetricsManager metricsManager, SkyWalkerCacheServices skyWalkerCacheServices,
|
||||
NacosServerHolder nacosServerHolder, ConsulServerHolder consulServerHolder) {
|
||||
this.metricsManager = metricsManager;
|
||||
|
||||
public NacosSyncToConsulServiceImpl(SkyWalkerCacheServices skyWalkerCacheServices,
|
||||
ConsulServerHolder consulServerHolder) {
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
this.consulServerHolder = consulServerHolder;
|
||||
}
|
||||
|
||||
|
||||
|
||||
@Override
|
||||
public boolean delete(TaskDO taskDO) {
|
||||
try {
|
||||
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
||||
|
||||
// 删除目标集群中同步的实例列表
|
||||
Response<List<HealthService>> serviceResponse =
|
||||
consulClient.getHealthServices(taskDO.getServiceName(), true, QueryParams.DEFAULT);
|
||||
List<HealthService> healthServices = serviceResponse.getValue();
|
||||
for (HealthService healthService : healthServices) {
|
||||
|
||||
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)) {
|
||||
consulClient.agentServiceDeregister(URLEncoder
|
||||
.encode(healthService.getService().getId(), StandardCharsets.UTF_8.name()));
|
||||
}
|
||||
public String composeInstanceKey(String ip, int port) {
|
||||
return String.join(":", ip, String.valueOf(port));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void register(TaskDO taskDO, Instance instance) {
|
||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId());
|
||||
consulClient.agentServiceRegister(buildSyncInstance(instance, taskDO));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void deregisterInstance(TaskDO taskDO) throws Exception {
|
||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId());
|
||||
Response<List<HealthService>> serviceResponse = consulClient.getHealthServices(taskDO.getServiceName(), true,
|
||||
QueryParams.DEFAULT);
|
||||
List<HealthService> healthServices = serviceResponse.getValue();
|
||||
for (HealthService healthService : healthServices) {
|
||||
|
||||
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)) {
|
||||
consulClient.agentServiceDeregister(
|
||||
URLEncoder.encode(healthService.getService().getId(), StandardCharsets.UTF_8));
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("delete a task from nacos to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
try {
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
|
||||
nacosListenerMap.putIfAbsent(taskDO.getTaskId(), event -> {
|
||||
if (event instanceof NamingEvent) {
|
||||
try {
|
||||
Set<String> instanceKeySet = new HashSet();
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName());
|
||||
// 先将新的注册一遍
|
||||
for (Instance instance : sourceInstances) {
|
||||
if (needSync(instance.getMetadata())) {
|
||||
consulClient.agentServiceRegister(buildSyncInstance(instance, taskDO));
|
||||
instance.getInstanceId();
|
||||
instanceKeySet.add(composeInstanceKey(instance.getIp(), instance.getPort()));
|
||||
}
|
||||
}
|
||||
|
||||
// 再将不存在的删掉
|
||||
Response<List<HealthService>> serviceResponse =
|
||||
consulClient.getHealthServices(taskDO.getServiceName(), true, QueryParams.DEFAULT);
|
||||
List<HealthService> healthServices = serviceResponse.getValue();
|
||||
for (HealthService healthService : healthServices) {
|
||||
|
||||
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)
|
||||
&& !instanceKeySet.contains(composeInstanceKey(healthService.getService().getAddress(),
|
||||
healthService.getService().getPort()))) {
|
||||
consulClient.agentServiceDeregister(URLEncoder
|
||||
.encode(healthService.getService().getId(), StandardCharsets.UTF_8.toString()));
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("event process fail, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
sourceNamingService.subscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
||||
} catch (Exception e) {
|
||||
log.error("sync task from nacos to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
return false;
|
||||
public void removeInvalidInstance(TaskDO taskDO, Set<String> invalidInstanceKeys)
|
||||
throws UnsupportedEncodingException {
|
||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId());
|
||||
Response<List<HealthService>> serviceResponse = consulClient.getHealthServices(taskDO.getServiceName(), true,
|
||||
QueryParams.DEFAULT);
|
||||
List<HealthService> healthServices = serviceResponse.getValue();
|
||||
for (HealthService healthService : healthServices) {
|
||||
|
||||
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)
|
||||
&& !invalidInstanceKeys.contains(composeInstanceKey(healthService.getService().getAddress(),
|
||||
healthService.getService().getPort()))) {
|
||||
consulClient.agentServiceDeregister(
|
||||
URLEncoder.encode(healthService.getService().getId(), StandardCharsets.UTF_8));
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private String composeInstanceKey(String ip, int port) {
|
||||
return ip + ":" + port;
|
||||
}
|
||||
|
||||
|
||||
public NewService buildSyncInstance(Instance instance, TaskDO taskDO) {
|
||||
NewService newService = new NewService();
|
||||
newService.setAddress(instance.getIp());
|
||||
|
@ -158,13 +109,13 @@ public class NacosSyncToConsulServiceImpl implements SyncService {
|
|||
newService.setId(instance.getInstanceId());
|
||||
List<String> tags = Lists.newArrayList();
|
||||
tags.addAll(instance.getMetadata().entrySet().stream()
|
||||
.map(entry -> String.join("=", entry.getKey(), entry.getValue())).collect(Collectors.toList()));
|
||||
tags.add(String.join("=", SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId()));
|
||||
tags.add(String.join("=", SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode()));
|
||||
tags.add(String.join("=", SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId()));
|
||||
.map(entry -> String.join(DELIMITER, entry.getKey(), entry.getValue())).collect(Collectors.toList()));
|
||||
tags.add(String.join(DELIMITER, SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId()));
|
||||
tags.add(String.join(DELIMITER, SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode()));
|
||||
tags.add(String.join(DELIMITER, SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId()));
|
||||
newService.setTags(tags);
|
||||
return newService;
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -10,23 +10,17 @@
|
|||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.extension.impl;
|
||||
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
import com.alibaba.nacos.api.naming.listener.Event;
|
||||
import com.alibaba.nacos.api.naming.listener.EventListener;
|
||||
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
||||
import com.alibaba.nacos.api.naming.pojo.Instance;
|
||||
import com.alibaba.nacos.common.utils.CollectionUtils;
|
||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||
import com.alibaba.nacossync.extension.SyncService;
|
||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||
import com.alibaba.nacossync.extension.eureka.EurekaNamingService;
|
||||
import com.alibaba.nacossync.extension.holder.EurekaServerHolder;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.netflix.appinfo.DataCenterInfo;
|
||||
import com.netflix.appinfo.InstanceInfo;
|
||||
|
@ -34,171 +28,96 @@ import com.netflix.appinfo.LeaseInfo;
|
|||
import com.netflix.appinfo.MyDataCenterInfo;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.util.*;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* @author zhanglong
|
||||
*/
|
||||
@Slf4j
|
||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.EUREKA)
|
||||
public class NacosSyncToEurekaServiceImpl implements SyncService {
|
||||
private final Map<String, EventListener> nacosListenerMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final MetricsManager metricsManager;
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
public class NacosSyncToEurekaServiceImpl extends AbstractNacosSync {
|
||||
|
||||
|
||||
private final EurekaServerHolder eurekaServerHolder;
|
||||
|
||||
public NacosSyncToEurekaServiceImpl(MetricsManager metricsManager, SkyWalkerCacheServices skyWalkerCacheServices,
|
||||
NacosServerHolder nacosServerHolder, EurekaServerHolder eurekaServerHolder) {
|
||||
this.metricsManager = metricsManager;
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
public NacosSyncToEurekaServiceImpl(EurekaServerHolder eurekaServerHolder,
|
||||
SkyWalkerCacheServices skyWalkerCacheServices) {
|
||||
this.eurekaServerHolder = eurekaServerHolder;
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
}
|
||||
|
||||
|
||||
|
||||
@Override
|
||||
public boolean delete(TaskDO taskDO) {
|
||||
try {
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
EurekaNamingService destNamingService =
|
||||
eurekaServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
||||
// 删除目标集群中同步的实例列表
|
||||
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
||||
if (allInstances != null) {
|
||||
for (InstanceInfo instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||
destNamingService.deregisterInstance(instance);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("delete task from nacos to eureka was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
try {
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
EurekaNamingService destNamingService =
|
||||
eurekaServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
|
||||
nacosListenerMap.putIfAbsent(taskDO.getTaskId(), event -> {
|
||||
processNamingEvent(taskDO, sourceNamingService, destNamingService, event);
|
||||
});
|
||||
|
||||
sourceNamingService.subscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
||||
} catch (Exception e) {
|
||||
log.error("sync task from eureka to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private void processNamingEvent(TaskDO taskDO, NamingService sourceNamingService,
|
||||
EurekaNamingService destNamingService, Event event) {
|
||||
if (event instanceof NamingEvent) {
|
||||
try {
|
||||
Set<String> instanceKeySet = new HashSet<>();
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName());
|
||||
// 先将新的注册一遍
|
||||
addAllNewInstance(taskDO, destNamingService, instanceKeySet, sourceInstances);
|
||||
// 再将不存在的删掉
|
||||
ifNecessaryDelete(taskDO, destNamingService, instanceKeySet);
|
||||
} catch (Exception e) {
|
||||
log.error("event process fail, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void ifNecessaryDelete(TaskDO taskDO, EurekaNamingService destNamingService, Set<String> instanceKeySet) {
|
||||
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
||||
if (allInstances != null){
|
||||
for (InstanceInfo instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO) && !instanceKeySet.contains(composeInstanceKey(instance.getIPAddr(),
|
||||
instance.getPort()))) {
|
||||
destNamingService.deregisterInstance(instance);
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void addAllNewInstance(TaskDO taskDO, EurekaNamingService destNamingService, Set<String> instanceKeySet,
|
||||
List<Instance> sourceInstances) {
|
||||
for (Instance instance : sourceInstances) {
|
||||
if (needSync(instance.getMetadata())) {
|
||||
destNamingService.registerInstance(buildSyncInstance(instance, taskDO));
|
||||
instanceKeySet.add(composeInstanceKey(instance.getIp(), instance.getPort()));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private String composeInstanceKey(String ip, int port) {
|
||||
public String composeInstanceKey(String ip, int port) {
|
||||
return ip + ":" + port;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void register(TaskDO taskDO, Instance instance) {
|
||||
EurekaNamingService destNamingService = eurekaServerHolder.get(taskDO.getDestClusterId());
|
||||
destNamingService.registerInstance(buildSyncInstance(instance, taskDO));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void deregisterInstance(TaskDO taskDO) {
|
||||
EurekaNamingService destNamingService = eurekaServerHolder.get(taskDO.getDestClusterId());
|
||||
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
||||
if (allInstances != null) {
|
||||
for (InstanceInfo instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||
destNamingService.deregisterInstance(instance);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void removeInvalidInstance(TaskDO taskDO, Set<String> invalidInstanceKeys) {
|
||||
EurekaNamingService destNamingService = eurekaServerHolder.get(taskDO.getDestClusterId());
|
||||
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
||||
if (CollectionUtils.isNotEmpty(allInstances)) {
|
||||
for (InstanceInfo instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO) && invalidInstanceKeys.contains(
|
||||
composeInstanceKey(instance.getIPAddr(), instance.getPort()))) {
|
||||
destNamingService.deregisterInstance(instance);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
private InstanceInfo buildSyncInstance(Instance instance, TaskDO taskDO) {
|
||||
DataCenterInfo dataCenterInfo = new MyDataCenterInfo(DataCenterInfo.Name.MyOwn);
|
||||
final Map<String, String> instanceMetadata = instance.getMetadata();
|
||||
HashMap<String, String> metadata = new HashMap<>(16);
|
||||
metadata.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
||||
metadata.put(SkyWalkerConstants.SYNC_SOURCE_KEY, skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metadata.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
||||
metadata.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||
metadata.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metadata.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||
metadata.putAll(instanceMetadata);
|
||||
String homePageUrl = obtainHomePageUrl(instance, instanceMetadata);
|
||||
String serviceName = taskDO.getServiceName();
|
||||
|
||||
return new InstanceInfo(
|
||||
instance.getIp() + ":" + serviceName + ":" + instance.getPort(),
|
||||
serviceName,
|
||||
null,
|
||||
instance.getIp(),
|
||||
null,
|
||||
new InstanceInfo.PortWrapper(true, instance.getPort()),
|
||||
null,
|
||||
homePageUrl+"/actuator/env",
|
||||
homePageUrl + "/actuator/info",
|
||||
homePageUrl + "/actuator/health",
|
||||
null,
|
||||
serviceName,
|
||||
serviceName,
|
||||
1,
|
||||
dataCenterInfo,
|
||||
instance.getIp(),
|
||||
InstanceInfo.InstanceStatus.UP,
|
||||
InstanceInfo.InstanceStatus.UNKNOWN,
|
||||
null,
|
||||
new LeaseInfo(30, 90,
|
||||
0L, 0L, 0L, 0L, 0L),
|
||||
false,
|
||||
metadata,
|
||||
System.currentTimeMillis(),
|
||||
System.currentTimeMillis(),
|
||||
null,
|
||||
null
|
||||
);
|
||||
|
||||
return new InstanceInfo(instance.getIp() + ":" + serviceName + ":" + instance.getPort(), serviceName, null,
|
||||
instance.getIp(), null, new InstanceInfo.PortWrapper(true, instance.getPort()), null,
|
||||
homePageUrl + "/actuator/env", homePageUrl + "/actuator/info", homePageUrl + "/actuator/health", null,
|
||||
serviceName, serviceName, 1, dataCenterInfo, instance.getIp(), InstanceInfo.InstanceStatus.UP,
|
||||
InstanceInfo.InstanceStatus.UNKNOWN, null, new LeaseInfo(30, 90, 0L, 0L, 0L, 0L, 0L), false, metadata,
|
||||
System.currentTimeMillis(), System.currentTimeMillis(), null, null);
|
||||
}
|
||||
|
||||
|
||||
private String obtainHomePageUrl(Instance instance, Map<String, String> instanceMetadata) {
|
||||
final String managementContextPath =
|
||||
obtainManagementContextPath(instanceMetadata);
|
||||
final String managementContextPath = obtainManagementContextPath(instanceMetadata);
|
||||
final String managementPort = instanceMetadata.getOrDefault(SkyWalkerConstants.MANAGEMENT_PORT_KEY,
|
||||
String.valueOf(instance.getPort()));
|
||||
return String.format("http://%s:%s%s",instance.getIp(),managementPort,managementContextPath);
|
||||
String.valueOf(instance.getPort()));
|
||||
return String.format("http://%s:%s%s", instance.getIp(), managementPort, managementContextPath);
|
||||
}
|
||||
|
||||
|
||||
private String obtainManagementContextPath(Map<String, String> instanceMetadata) {
|
||||
final String path = instanceMetadata.getOrDefault(SkyWalkerConstants.MANAGEMENT_CONTEXT_PATH_KEY, "");
|
||||
if (path.endsWith("/")) {
|
||||
|
@ -206,7 +125,6 @@ public class NacosSyncToEurekaServiceImpl implements SyncService {
|
|||
}
|
||||
return path;
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -10,6 +10,7 @@
|
|||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.extension.impl;
|
||||
|
||||
import com.alibaba.nacos.api.exception.NacosException;
|
||||
|
@ -22,23 +23,32 @@ import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
|||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.extension.SyncService;
|
||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.Collections;
|
||||
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||
import com.alibaba.nacossync.util.StringUtils;
|
||||
import com.google.common.base.Stopwatch;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.DisposableBean;
|
||||
import org.springframework.beans.factory.InitializingBean;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collection;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.TreeSet;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.stream.Collectors;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
import static com.alibaba.nacossync.constant.SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY;
|
||||
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||
|
||||
/**
|
||||
* @author yangyshdan
|
||||
|
@ -47,91 +57,173 @@ import org.springframework.beans.factory.annotation.Autowired;
|
|||
|
||||
@Slf4j
|
||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.NACOS)
|
||||
public class NacosSyncToNacosServiceImpl implements SyncService {
|
||||
|
||||
private Map<String, EventListener> listenerMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final Map<String, Set<String>> sourceInstanceSnapshot = new ConcurrentHashMap<>();
|
||||
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
|
||||
@Autowired
|
||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
@Autowired
|
||||
private NacosServerHolder nacosServerHolder;
|
||||
|
||||
public class NacosSyncToNacosServiceImpl implements SyncService, InitializingBean, DisposableBean {
|
||||
|
||||
private final Map<String, EventListener> listenerMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final Map<String, Integer> syncTaskTap = new ConcurrentHashMap<>();
|
||||
|
||||
private final ConcurrentHashMap<String, TaskDO> allSyncTaskMap = new ConcurrentHashMap<>();
|
||||
|
||||
private ScheduledExecutorService executorService;
|
||||
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
public NacosSyncToNacosServiceImpl(MetricsManager metricsManager, NacosServerHolder nacosServerHolder,
|
||||
ClusterAccessService clusterAccessService, SkyWalkerCacheServices skyWalkerCacheServices) {
|
||||
this.metricsManager = metricsManager;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
}
|
||||
|
||||
/**
|
||||
* Due to network issues or other reasons, the Nacos Sync synchronization tasks may fail,
|
||||
* resulting in the target cluster's registry missing synchronized instances.
|
||||
* To prevent the target cluster's registry from missing synchronized instances for an extended period,
|
||||
* a fallback worker thread is started every 5 minutes to execute all synchronization tasks.
|
||||
*/
|
||||
@Override
|
||||
public void afterPropertiesSet() {
|
||||
initializeExecutorService();
|
||||
scheduleSyncTasks();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void destroy() {
|
||||
if (executorService != null && !executorService.isShutdown()) {
|
||||
executorService.shutdown();
|
||||
try {
|
||||
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||
executorService.shutdownNow();
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
executorService.shutdownNow();
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void initializeExecutorService() {
|
||||
executorService = Executors.newSingleThreadScheduledExecutor(r -> {
|
||||
Thread t = new Thread(r);
|
||||
t.setDaemon(true);
|
||||
t.setName("com.alibaba.nacossync.basic.synctask");
|
||||
return t;
|
||||
});
|
||||
}
|
||||
|
||||
private void scheduleSyncTasks() {
|
||||
executorService.scheduleWithFixedDelay(this::executeSyncTasks, 0, 300, TimeUnit.SECONDS);
|
||||
}
|
||||
|
||||
private void executeSyncTasks() {
|
||||
if (allSyncTaskMap.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
|
||||
Collection<TaskDO> taskCollections = allSyncTaskMap.values();
|
||||
List<TaskDO> taskDOList = new ArrayList<>(taskCollections);
|
||||
|
||||
if (CollectionUtils.isNotEmpty(taskDOList)) {
|
||||
BatchTaskExecutor.batchOperation(taskDOList, this::executeTask);
|
||||
}
|
||||
}
|
||||
|
||||
private void executeTask(TaskDO task) {
|
||||
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||
String taskId = task.getTaskId();
|
||||
try {
|
||||
NamingService sourceNamingService = nacosServerHolder.get(task.getSourceClusterId());
|
||||
NamingService destNamingService = nacosServerHolder.get(task.getDestClusterId());
|
||||
doSync(taskId, task, sourceNamingService, destNamingService);
|
||||
} catch (NacosException e) {
|
||||
log.error("sync task from nacos to nacos failed, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
} catch (Exception e) {
|
||||
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
} finally {
|
||||
stopwatch.stop();
|
||||
log.debug("Task execution time for taskId {}: {} ms", taskId, stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean delete(TaskDO taskDO) {
|
||||
try {
|
||||
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
//移除订阅
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), listenerMap.remove(taskDO.getTaskId()));
|
||||
sourceInstanceSnapshot.remove(taskDO.getTaskId());
|
||||
|
||||
// 删除目标集群中同步的实例列表
|
||||
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||
int level = clusterAccessService.findClusterLevel(taskDO.getSourceClusterId());
|
||||
String taskId = taskDO.getTaskId();
|
||||
|
||||
//Handle individual service
|
||||
if (StringUtils.isEmpty(taskId)) {
|
||||
log.warn("taskId is null data synchronization is not currently performed.{}", taskId);
|
||||
return false;
|
||||
}
|
||||
EventListener listener = listenerMap.remove(taskId);
|
||||
if (listener!= null) {
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()), listener);
|
||||
}
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
new ArrayList<>(), false);
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), false);
|
||||
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
for (Instance instance : sourceInstances) {
|
||||
if (needSync(instance.getMetadata())) {
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
||||
if (needSync(instance.getMetadata(), level, taskDO.getDestClusterId())) {
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||
}
|
||||
}
|
||||
// Remove all tasks that need to be synchronized.
|
||||
allSyncTaskMap.remove(taskId);
|
||||
|
||||
} catch (Exception e) {
|
||||
log.error("delete task from nacos to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
log.error("delete task from nacos to nacos was failed, operationalId:{}", taskDO.getOperationId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
public boolean sync(TaskDO taskDO, Integer index) {
|
||||
log.info("Thread {} started synchronization at {}", Thread.currentThread().getId(), System.currentTimeMillis());
|
||||
String taskId = taskDO.getTaskId();
|
||||
try {
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
|
||||
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
allSyncTaskMap.put(taskId, taskDO);
|
||||
// To prevent issues where tasks paused for synchronization, newly created tasks after deletion,
|
||||
// or resynchronization tasks do not receive new events and hence cannot synchronize,
|
||||
// perform a full synchronization of tasks before subscribing to events each time.
|
||||
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||
doSync(taskId, taskDO, sourceNamingService, destNamingService);
|
||||
log.debug("Time taken to synchronize a service registration: {} ms",
|
||||
stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||
this.listenerMap.putIfAbsent(taskId, event -> {
|
||||
if (event instanceof NamingEvent) {
|
||||
NamingEvent namingEvent = (NamingEvent) event;
|
||||
log.info("Detected changes in service {} information, taskId: {}, number of instances: {}, initiating synchronization",
|
||||
namingEvent.getServiceName(), taskId,
|
||||
namingEvent.getInstances() == null ? null : namingEvent.getInstances().size());
|
||||
try {
|
||||
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
new ArrayList<>(), false);
|
||||
// 先删除不存在的
|
||||
this.removeInvalidInstance(taskDO, taskId, destNamingService, sourceInstances);
|
||||
Set<String> latestSyncInstance = new TreeSet<>();
|
||||
//再次添加新实例
|
||||
Set<String> instanceKeys = sourceInstanceSnapshot.get(taskId);
|
||||
for (Instance instance : sourceInstances) {
|
||||
if (needSync(instance.getMetadata())) {
|
||||
String instanceKey = composeInstanceKey(instance);
|
||||
if (CollectionUtils.isEmpty(instanceKeys) || !instanceKeys.contains(instanceKey)) {
|
||||
destNamingService.registerInstance(taskDO.getServiceName(),
|
||||
buildSyncInstance(instance, taskDO));
|
||||
}
|
||||
latestSyncInstance.add(instanceKey);
|
||||
|
||||
}
|
||||
}
|
||||
if (CollectionUtils.isNotEmpty(latestSyncInstance)) {
|
||||
|
||||
log.info("任务Id:{},已同步实例个数:{}", taskId, latestSyncInstance.size());
|
||||
sourceInstanceSnapshot.put(taskId, latestSyncInstance);
|
||||
}
|
||||
doSync(taskId, taskDO, sourceNamingService, destNamingService);
|
||||
log.info("Detected synchronization end for service {}", namingEvent.getServiceName());
|
||||
} catch (Exception e) {
|
||||
log.error("event process fail, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
sourceNamingService.subscribe(taskDO.getServiceName(), listenerMap.get(taskId));
|
||||
sourceNamingService.subscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
listenerMap.get(taskId));
|
||||
} catch (Exception e) {
|
||||
log.error("sync task from nacos to nacos was failed, taskId:{}", taskId, e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
|
@ -139,31 +231,192 @@ public class NacosSyncToNacosServiceImpl implements SyncService {
|
|||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private void removeInvalidInstance(TaskDO taskDO, String taskId, NamingService destNamingService,
|
||||
List<Instance> sourceInstances) throws NacosException {
|
||||
if (this.sourceInstanceSnapshot.containsKey(taskId)) {
|
||||
Set<String> oldInstanceKeys = this.sourceInstanceSnapshot.get(taskId);
|
||||
List<String> newInstanceKeys = sourceInstances.stream().map(this::composeInstanceKey)
|
||||
.collect(Collectors.toList());
|
||||
Collection<String> instanceKeys = Collections.subtract(oldInstanceKeys, newInstanceKeys);
|
||||
for (String instanceKey : instanceKeys) {
|
||||
log.info("任务Id:{},移除无效同步实例:{}", taskId, instanceKey);
|
||||
String[] split = instanceKey.split(":", -1);
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), split[0],
|
||||
Integer.parseInt(split[1]));
|
||||
|
||||
|
||||
|
||||
private void doSync(String taskId, TaskDO taskDO, NamingService sourceNamingService,
|
||||
NamingService destNamingService) throws NacosException {
|
||||
if (syncTaskTap.putIfAbsent(taskId, 1) != null) {
|
||||
log.info("Task ID:{} - the previous synchronization task has not finished yet", taskId);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||
|
||||
int level = clusterAccessService.findClusterLevel(taskDO.getSourceClusterId());
|
||||
if (CollectionUtils.isNotEmpty(sourceInstances) && sourceInstances.get(0).isEphemeral()) {
|
||||
// Handle batch data synchronization of ephemeral instances, need to get all current service instances.
|
||||
// TODO: When the Client is version 1.x, execute the same synchronization method as persistent instances.
|
||||
handlerPersistenceInstance(taskDO, destNamingService, sourceInstances, level);
|
||||
} else if (CollectionUtils.isEmpty(sourceInstances)) {
|
||||
// If the current source cluster is empty, then directly deregister the instances in the target cluster.
|
||||
log.debug("service {} needs to sync ephemeral instance num is null: serviceName ", taskDO.getServiceName());
|
||||
processDeRegisterInstances(taskDO, destNamingService);
|
||||
} else {
|
||||
// Handle batch data synchronization of persistent instances.
|
||||
handlerPersistenceInstance(taskDO, destNamingService, sourceInstances, level);
|
||||
}
|
||||
} finally {
|
||||
syncTaskTap.remove(taskId);
|
||||
}
|
||||
}
|
||||
|
||||
private void handlerPersistenceInstance(TaskDO taskDO, NamingService destNamingService,
|
||||
List<Instance> sourceInstances, int level) throws NacosException {
|
||||
List<Instance> needRegisterInstance = new ArrayList<>();
|
||||
for (Instance instance : sourceInstances) {
|
||||
if (needSync(instance.getMetadata(), level, taskDO.getDestClusterId())) {
|
||||
needRegisterInstance.add(buildSyncInstance(instance, taskDO));
|
||||
}
|
||||
}
|
||||
List<Instance> destAllInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||
|
||||
// Get the instances that the destination cluster has already synchronized
|
||||
List<Instance> destHasSyncInstances = destAllInstances.stream()
|
||||
.filter(instance -> hasSync(instance, taskDO.getSourceClusterId())).collect(Collectors.toList());
|
||||
|
||||
// The following two conversions are necessary because the Nacos Instance's equals method
|
||||
// is flawed and cannot be used for direct comparison.
|
||||
// The reason is that Instance's equals method compares Metadata using the toString method,
|
||||
// and Metadata is of type HashMap, which does not guarantee order in its toString representation.
|
||||
|
||||
// Convert destHasSyncInstances to a Map with the concatenated string as key
|
||||
Map<String, Instance> destInstanceMap = destHasSyncInstances.stream()
|
||||
.collect(Collectors.toMap(NacosSyncToNacosServiceImpl::getInstanceKey, instance -> instance));
|
||||
|
||||
// Convert newInstances to a Map with the concatenated string as key
|
||||
Map<String, Instance> needRegisterMap = needRegisterInstance.stream()
|
||||
.collect(Collectors.toMap(NacosSyncToNacosServiceImpl::getInstanceKey, instance -> instance));
|
||||
|
||||
// Remove instances from newInstanceMap that are present in destInstanceMap
|
||||
List<Instance> newInstances = removeSyncedInstances(destInstanceMap, needRegisterMap);
|
||||
|
||||
// Remove instances from destInstanceMap that are present in newInstanceMap
|
||||
List<Instance> invalidInstances = getInvalidInstances(destInstanceMap, needRegisterMap);
|
||||
|
||||
// Register each instance one by one. Take one instance at a time.
|
||||
for (Instance newInstance : newInstances) {
|
||||
destNamingService.registerInstance(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
newInstance);
|
||||
}
|
||||
|
||||
if (CollectionUtils.isNotEmpty(invalidInstances)) {
|
||||
log.info("taskId: {}, service {} deregistered, number of executions: {}", taskDO.getTaskId(), taskDO.getServiceName(),
|
||||
destHasSyncInstances.size());
|
||||
}
|
||||
for (Instance instance : invalidInstances) {
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
instance);
|
||||
}
|
||||
}
|
||||
|
||||
private List<Instance> getInvalidInstances(Map<String, Instance> destInstanceMap, Map<String, Instance> needRegisterMap) {
|
||||
Map<String, Instance> destClone = new HashMap<>(destInstanceMap);
|
||||
|
||||
// Convert newInstances to a Map with the concatenated string as key
|
||||
Map<String, Instance> needRegisterClone = new HashMap<>(needRegisterMap);
|
||||
|
||||
// Remove instances from newInstanceMap that are present in destInstanceMap
|
||||
destClone.keySet().removeAll(needRegisterClone.keySet());
|
||||
|
||||
return new ArrayList<>(destClone.values());
|
||||
}
|
||||
|
||||
|
||||
public List<Instance> removeSyncedInstances(Map<String, Instance> destInstanceMap, Map<String, Instance> needRegisterMap) {
|
||||
// Convert destHasSyncInstances to a Map with the concatenated string as key
|
||||
|
||||
Map<String, Instance> destClone = new HashMap<>(destInstanceMap);
|
||||
|
||||
// Convert newInstances to a Map with the concatenated string as key
|
||||
Map<String, Instance> needRegisterClone = new HashMap<>(needRegisterMap);
|
||||
|
||||
// Remove instances from newInstanceMap that are present in destInstanceMap
|
||||
needRegisterClone.keySet().removeAll(destClone.keySet());
|
||||
|
||||
return new ArrayList<>(needRegisterClone.values());
|
||||
}
|
||||
|
||||
|
||||
private boolean hasSync(Instance instance, String sourceClusterId) {
|
||||
if (instance.getMetadata() != null) {
|
||||
String sourceClusterKey = instance.getMetadata().get(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY);
|
||||
return sourceClusterKey != null && sourceClusterKey.equals(sourceClusterId);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* When the number of instances that the source cluster needs to synchronize is 0,
|
||||
* if the target cluster still has instances synchronized with the source cluster,
|
||||
* perform unregistration.
|
||||
*
|
||||
* @param destNamingService Destination cluster naming service
|
||||
* @throws NacosException
|
||||
*/
|
||||
private void processDeRegisterInstances(TaskDO taskDO, NamingService destNamingService) throws NacosException {
|
||||
// If the instances in sourceInstances are empty, it means the instances are offline or do not exist.
|
||||
List<Instance> destInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), false);
|
||||
// If the instances in the target cluster are also empty, then no operation is needed.
|
||||
if (CollectionUtils.isEmpty(destInstances)) {
|
||||
return;
|
||||
}
|
||||
destInstances = filterInstancesForRemoval(destInstances, taskDO.getSourceClusterId());
|
||||
if (CollectionUtils.isNotEmpty(destInstances)) {
|
||||
// Deregister each instance one by one. Take one instance at a time.
|
||||
// Need to handle redo, otherwise, it will be registered again.
|
||||
for (Instance destInstance : destInstances) {
|
||||
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), destInstance);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private String composeInstanceKey(Instance instance) {
|
||||
return instance.getIp() + ":" + instance.getPort();
|
||||
|
||||
private List<Instance> filterInstancesForRemoval(List<Instance> destInstances, String sourceClusterId) {
|
||||
return destInstances.stream().filter(instance -> !instance.getMetadata().isEmpty())
|
||||
.filter(instance -> needDeregister(instance.getMetadata().get(SOURCE_CLUSTER_ID_KEY), sourceClusterId))
|
||||
.collect(Collectors.toList());
|
||||
|
||||
}
|
||||
|
||||
|
||||
public Instance buildSyncInstance(Instance instance, TaskDO taskDO) {
|
||||
|
||||
private boolean needDeregister(String destClusterId, String sourceClusterId) {
|
||||
if (!StringUtils.isEmpty(destClusterId)) {
|
||||
return destClusterId.equals(sourceClusterId);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private boolean needSync(Map<String, String> sourceMetaData, int level, String destClusterId) {
|
||||
// Regular cluster (default)
|
||||
if (level == 0) {
|
||||
return SyncService.super.needSync(sourceMetaData);
|
||||
}
|
||||
// Central cluster, as long as the instance is not from the target cluster,
|
||||
// it needs to be synchronized (extended functionality)
|
||||
return !destClusterId.equals(sourceMetaData.get(SOURCE_CLUSTER_ID_KEY));
|
||||
}
|
||||
|
||||
|
||||
private Instance buildSyncInstance(Instance instance, TaskDO taskDO) {
|
||||
Instance temp = getInstance(instance);
|
||||
temp.addMetadata(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||
temp.addMetadata(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
temp.addMetadata(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||
//The flag is a synchronous instance
|
||||
temp.addMetadata(SkyWalkerConstants.SYNC_INSTANCE_TAG,
|
||||
taskDO.getSourceClusterId() + "@@" + taskDO.getVersion());
|
||||
return temp;
|
||||
}
|
||||
|
||||
private static Instance getInstance(Instance instance) {
|
||||
Instance temp = new Instance();
|
||||
temp.setInstanceId(instance.getInstanceId());
|
||||
temp.setIp(instance.getIp());
|
||||
temp.setPort(instance.getPort());
|
||||
temp.setClusterName(instance.getClusterName());
|
||||
|
@ -172,15 +425,19 @@ public class NacosSyncToNacosServiceImpl implements SyncService {
|
|||
temp.setHealthy(instance.isHealthy());
|
||||
temp.setWeight(instance.getWeight());
|
||||
temp.setEphemeral(instance.isEphemeral());
|
||||
Map<String, String> metaData = new HashMap<>();
|
||||
metaData.putAll(instance.getMetadata());
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
||||
Map<String, String> metaData = new HashMap<>(instance.getMetadata());
|
||||
temp.setMetadata(metaData);
|
||||
return temp;
|
||||
}
|
||||
|
||||
|
||||
|
||||
private static String getInstanceKey(Instance instance) {
|
||||
return String.join("|",
|
||||
instance.getIp(),
|
||||
String.valueOf(instance.getPort()),
|
||||
String.valueOf(instance.getWeight()),
|
||||
String.valueOf(instance.isHealthy()),
|
||||
String.valueOf(instance.isEphemeral()),
|
||||
instance.getClusterName(),
|
||||
instance.getServiceName());
|
||||
}
|
||||
}
|
||||
|
|
|
@ -12,9 +12,6 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.extension.impl;
|
||||
|
||||
import static com.alibaba.nacossync.util.StringUtils.convertDubboFullPathForZk;
|
||||
import static com.alibaba.nacossync.util.StringUtils.convertDubboProvidersPath;
|
||||
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
import com.alibaba.nacos.api.naming.listener.EventListener;
|
||||
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
||||
|
@ -32,20 +29,25 @@ import com.alibaba.nacossync.monitor.MetricsManager;
|
|||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.DubboConstants;
|
||||
import com.google.common.collect.Sets;
|
||||
import java.io.UnsupportedEncodingException;
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.curator.framework.CuratorFramework;
|
||||
import org.apache.curator.framework.recipes.cache.PathChildrenCache;
|
||||
import org.apache.curator.framework.recipes.cache.PathChildrenCacheEvent;
|
||||
import org.apache.curator.utils.CloseableUtils;
|
||||
import org.apache.zookeeper.CreateMode;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
import java.io.UnsupportedEncodingException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||
import static com.alibaba.nacossync.util.StringUtils.convertDubboFullPathForZk;
|
||||
import static com.alibaba.nacossync.util.StringUtils.convertDubboProvidersPath;
|
||||
|
||||
/**
|
||||
* Nacos 同步 Zk 数据
|
||||
|
@ -57,8 +59,7 @@ import org.springframework.beans.factory.annotation.Autowired;
|
|||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.ZK)
|
||||
public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
/**
|
||||
* @description The Nacos listener map.
|
||||
|
@ -90,12 +91,13 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
|||
|
||||
private final ZookeeperServerHolder zookeeperServerHolder;
|
||||
|
||||
@Autowired
|
||||
|
||||
public NacosSyncToZookeeperServiceImpl(SkyWalkerCacheServices skyWalkerCacheServices,
|
||||
NacosServerHolder nacosServerHolder, ZookeeperServerHolder zookeeperServerHolder) {
|
||||
NacosServerHolder nacosServerHolder, ZookeeperServerHolder zookeeperServerHolder, MetricsManager metricsManager) {
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
this.zookeeperServerHolder = zookeeperServerHolder;
|
||||
this.metricsManager = metricsManager;
|
||||
}
|
||||
|
||||
@Override
|
||||
|
@ -103,15 +105,18 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
|||
try {
|
||||
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||
EventListener eventListener = nacosListenerMap.remove(taskDO.getTaskId());
|
||||
PathChildrenCache pathChildrenCache = pathChildrenCacheMap.get(taskDO.getTaskId());
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), eventListener);
|
||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
eventListener);
|
||||
CloseableUtils.closeQuietly(pathChildrenCache);
|
||||
Set<String> instanceUrlSet = instanceBackupMap.get(taskDO.getTaskId());
|
||||
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
for (String instanceUrl : instanceUrlSet) {
|
||||
client.delete().quietly().forPath(instanceUrl);
|
||||
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId());
|
||||
if(!instanceUrlSet.isEmpty()){
|
||||
for (String instanceUrl : instanceUrlSet) {
|
||||
client.delete().quietly().forPath(instanceUrl);
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("delete task from nacos to zk was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
|
@ -122,23 +127,24 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
|||
}
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
public boolean sync(TaskDO taskDO, Integer index) {
|
||||
try {
|
||||
NamingService sourceNamingService =
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
||||
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
||||
nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId());
|
||||
nacosListenerMap.putIfAbsent(taskDO.getTaskId(), event -> {
|
||||
if (event instanceof NamingEvent) {
|
||||
try {
|
||||
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName());
|
||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||
Set<String> newInstanceUrlSet = getWaitingToAddInstance(taskDO, client, sourceInstances);
|
||||
|
||||
// 获取之前的备份 删除无效实例
|
||||
// fetch the instance backup
|
||||
deleteInvalidInstances(taskDO, client, newInstanceUrlSet);
|
||||
// 替换当前备份为最新备份
|
||||
// replace the instance backup
|
||||
instanceBackupMap.put(taskDO.getTaskId(), newInstanceUrlSet);
|
||||
// 尝试恢复因为zk客户端意外断开导致的实例数据
|
||||
// try to compensate for the removed instance
|
||||
tryToCompensate(taskDO, sourceNamingService, sourceInstances);
|
||||
} catch (Exception e) {
|
||||
log.error("event process fail, taskId:{}", taskDO.getTaskId(), e);
|
||||
|
@ -148,7 +154,8 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
|||
}
|
||||
});
|
||||
|
||||
sourceNamingService.subscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
||||
sourceNamingService.subscribe(taskDO.getServiceName(),getGroupNameOrDefault(taskDO.getGroupName()),
|
||||
nacosListenerMap.get(taskDO.getTaskId()));
|
||||
} catch (Exception e) {
|
||||
log.error("sync task from nacos to zk was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
|
@ -156,28 +163,45 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
|||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
private void tryToCompensate(TaskDO taskDO, NamingService sourceNamingService, List<Instance> sourceInstances) {
|
||||
if (!CollectionUtils.isEmpty(sourceInstances)) {
|
||||
final PathChildrenCache pathCache = getPathCache(taskDO);
|
||||
if (pathCache.getListenable().size() == 0) { // 防止重复注册
|
||||
pathCache.getListenable().addListener((zkClient, zkEvent) -> {
|
||||
if (zkEvent.getType() == PathChildrenCacheEvent.Type.CHILD_REMOVED) {
|
||||
List<Instance> allInstances =
|
||||
sourceNamingService.getAllInstances(taskDO.getServiceName());
|
||||
for (Instance instance : allInstances) {
|
||||
String instanceUrl = buildSyncInstance(instance, taskDO);
|
||||
String zkInstancePath = zkEvent.getData().getPath();
|
||||
if (zkInstancePath.equals(instanceUrl)) {
|
||||
zkClient.create().creatingParentsIfNeeded().withMode(CreateMode.EPHEMERAL)
|
||||
.forPath(zkInstancePath);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
// Avoiding re-registration if there is already a listener registered.
|
||||
if (pathCache.getListenable().size() == 0) {
|
||||
registerCompensationListener(pathCache, taskDO, sourceNamingService);
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
private void registerCompensationListener(PathChildrenCache pathCache, TaskDO taskDO, NamingService sourceNamingService) {
|
||||
pathCache.getListenable().addListener((zkClient, zkEvent) -> {
|
||||
try {
|
||||
if (zkEvent.getType() == PathChildrenCacheEvent.Type.CHILD_REMOVED) {
|
||||
compensateOnChildRemoval(zkClient, zkEvent, sourceNamingService, taskDO);
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("Error processing ZooKeeper event: {}", zkEvent.getType(), e);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private void compensateOnChildRemoval(CuratorFramework zkClient, PathChildrenCacheEvent zkEvent, NamingService sourceNamingService, TaskDO taskDO) {
|
||||
String zkInstancePath = null;
|
||||
try {
|
||||
List<Instance> allInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()));
|
||||
zkInstancePath = zkEvent.getData().getPath();
|
||||
for (Instance instance : allInstances) {
|
||||
String instanceUrl = buildSyncInstance(instance, taskDO);
|
||||
if (zkInstancePath.equals(instanceUrl)) {
|
||||
zkClient.create().creatingParentsIfNeeded().withMode(CreateMode.EPHEMERAL)
|
||||
.forPath(zkInstancePath);
|
||||
log.info("Compensated by re-creating the removed node at path: {}", zkInstancePath);
|
||||
break;
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("Failed to compensate for the removed node at path: {}", zkInstancePath, e);
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -210,12 +234,11 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
|||
}
|
||||
|
||||
protected String buildSyncInstance(Instance instance, TaskDO taskDO) throws UnsupportedEncodingException {
|
||||
Map<String, String> metaData = new HashMap<>();
|
||||
metaData.putAll(instance.getMetadata());
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
||||
Map<String, String> metaData = new HashMap<>(instance.getMetadata());
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||
|
||||
String servicePath = monitorPath.computeIfAbsent(taskDO.getTaskId(),
|
||||
key -> convertDubboProvidersPath(metaData.get(DubboConstants.INTERFACE_KEY)));
|
||||
|
@ -225,16 +248,16 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
|||
|
||||
|
||||
/**
|
||||
* 获取zk path child 监听缓存类
|
||||
* fetch zk path cache
|
||||
*
|
||||
* @param taskDO 任务对象
|
||||
* @return zk节点操作缓存对象
|
||||
* @param taskDO task instance
|
||||
* @return zk path cache
|
||||
*/
|
||||
private PathChildrenCache getPathCache(TaskDO taskDO) {
|
||||
return pathChildrenCacheMap.computeIfAbsent(taskDO.getTaskId(), (key) -> {
|
||||
try {
|
||||
PathChildrenCache pathChildrenCache = new PathChildrenCache(
|
||||
zookeeperServerHolder.get(taskDO.getDestClusterId(), ""), monitorPath.get(key), false);
|
||||
zookeeperServerHolder.get(taskDO.getDestClusterId()), monitorPath.get(key), false);
|
||||
pathChildrenCache.start(PathChildrenCache.StartMode.BUILD_INITIAL_CACHE);
|
||||
return pathChildrenCache;
|
||||
} catch (Exception e) {
|
||||
|
|
|
@ -10,22 +10,8 @@
|
|||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package com.alibaba.nacossync.extension.impl;
|
||||
|
||||
import static com.alibaba.nacossync.util.DubboConstants.ALL_SERVICE_NAME_PATTERN;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_PATH_FORMAT;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_ROOT_PATH;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.GROUP_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_IP_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_PORT_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INTERFACE_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.PROTOCOL_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.VERSION_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.WEIGHT_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.ZOOKEEPER_SEPARATOR;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.createServiceName;
|
||||
import static com.alibaba.nacossync.util.StringUtils.parseIpAndPortString;
|
||||
import static com.alibaba.nacossync.util.StringUtils.parseQueryString;
|
||||
package com.alibaba.nacossync.extension.impl;
|
||||
|
||||
import com.alibaba.nacos.api.exception.NacosException;
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
|
@ -40,20 +26,38 @@ import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
|||
import com.alibaba.nacossync.extension.holder.ZookeeperServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.function.Predicate;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.curator.framework.CuratorFramework;
|
||||
import org.apache.curator.framework.recipes.cache.TreeCache;
|
||||
import org.apache.curator.framework.recipes.cache.TreeCacheEvent;
|
||||
import org.apache.curator.utils.CloseableUtils;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
import static com.alibaba.nacossync.util.DubboConstants.ALL_SERVICE_NAME_PATTERN;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_PATH_FORMAT;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_ROOT_PATH;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.GROUP_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_IP_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_PORT_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INTERFACE_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.PROTOCOL_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.VERSION_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.WEIGHT_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.ZOOKEEPER_SEPARATOR;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.createServiceName;
|
||||
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||
import static com.alibaba.nacossync.util.StringUtils.parseIpAndPortString;
|
||||
import static com.alibaba.nacossync.util.StringUtils.parseQueryString;
|
||||
|
||||
/**
|
||||
* @author paderlol
|
||||
|
@ -63,59 +67,55 @@ import org.springframework.beans.factory.annotation.Autowired;
|
|||
@Slf4j
|
||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.ZK, destinationCluster = ClusterTypeEnum.NACOS)
|
||||
public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
||||
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
|
||||
|
||||
private static final String DEFAULT_WEIGHT = "1.0";
|
||||
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
/**
|
||||
* Listener cache of Zookeeper format taskId -> PathChildrenCache instance
|
||||
*/
|
||||
private Map<String, TreeCache> treeCacheMap = new ConcurrentHashMap<>();
|
||||
private final Map<String, TreeCache> treeCacheMap = new ConcurrentHashMap<>();
|
||||
|
||||
/**
|
||||
* service name cache
|
||||
*/
|
||||
private Map<String, String> nacosServiceNameMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final Map<String, String> nacosServiceNameMap = new ConcurrentHashMap<>();
|
||||
|
||||
private final ZookeeperServerHolder zookeeperServerHolder;
|
||||
|
||||
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
|
||||
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
@Autowired
|
||||
|
||||
|
||||
public ZookeeperSyncToNacosServiceImpl(ZookeeperServerHolder zookeeperServerHolder,
|
||||
NacosServerHolder nacosServerHolder, SkyWalkerCacheServices skyWalkerCacheServices) {
|
||||
NacosServerHolder nacosServerHolder, SkyWalkerCacheServices skyWalkerCacheServices,
|
||||
MetricsManager metricsManager) {
|
||||
this.zookeeperServerHolder = zookeeperServerHolder;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.metricsManager = metricsManager;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public boolean sync(TaskDO taskDO) {
|
||||
public boolean sync(TaskDO taskDO, Integer index) {
|
||||
try {
|
||||
if (treeCacheMap.containsKey(taskDO.getTaskId())) {
|
||||
return true;
|
||||
}
|
||||
|
||||
TreeCache treeCache = getTreeCache(taskDO);
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
||||
if (!initializeTreeCache(taskDO)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
if (destNamingService == null) {
|
||||
logAndRecordSyncError("Failed to obtain NamingService for destination clusterId: {}", taskDO.getDestClusterId(), null);
|
||||
return false;
|
||||
}
|
||||
// 初次执行任务统一注册所有实例
|
||||
registerAllInstances(taskDO, destNamingService);
|
||||
//注册ZK监听
|
||||
Objects.requireNonNull(treeCache).getListenable().addListener((client, event) -> {
|
||||
try {
|
||||
|
||||
String path = event.getData().getPath();
|
||||
Map<String, String> queryParam = parseQueryString(path);
|
||||
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
||||
processEvent(taskDO, destNamingService, event, path, queryParam);
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("event process from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
}
|
||||
|
||||
});
|
||||
setupListener(taskDO, destNamingService);
|
||||
} catch (Exception e) {
|
||||
log.error("sync task from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
|
@ -123,128 +123,105 @@ public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
|||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private void processEvent(TaskDO taskDO, NamingService destNamingService, TreeCacheEvent event, String path,
|
||||
Map<String, String> queryParam) throws NacosException {
|
||||
if(!com.alibaba.nacossync.util.StringUtils.isDubboProviderPath(path)) {
|
||||
return;
|
||||
}
|
||||
|
||||
Map<String, String> ipAndPortParam = parseIpAndPortString(path);
|
||||
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
||||
String serviceName = queryParam.get(INTERFACE_KEY);
|
||||
switch (event.getType()) {
|
||||
case NODE_ADDED:
|
||||
case NODE_UPDATED:
|
||||
|
||||
destNamingService.registerInstance(
|
||||
getServiceNameFromCache(serviceName, queryParam), instance);
|
||||
break;
|
||||
case NODE_REMOVED:
|
||||
|
||||
destNamingService.deregisterInstance(
|
||||
getServiceNameFromCache(serviceName, queryParam),
|
||||
ipAndPortParam.get(INSTANCE_IP_KEY),
|
||||
Integer.parseInt(ipAndPortParam.get(INSTANCE_PORT_KEY)));
|
||||
nacosServiceNameMap.remove(serviceName);
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private void registerAllInstances(TaskDO taskDO, NamingService destNamingService) throws Exception {
|
||||
CuratorFramework zk = zookeeperServerHolder.get(taskDO.getSourceClusterId(), "");
|
||||
if(!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
||||
registerALLInstances0(taskDO, destNamingService, zk, taskDO.getServiceName());
|
||||
} else {
|
||||
// 同步全部
|
||||
List<String> serviceList = zk.getChildren().forPath(DUBBO_ROOT_PATH);
|
||||
for(String serviceName : serviceList) {
|
||||
registerALLInstances0(taskDO, destNamingService, zk, serviceName);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void registerALLInstances0(TaskDO taskDO, NamingService destNamingService, CuratorFramework zk,
|
||||
String serviceName) throws Exception {
|
||||
String path = String.format(DUBBO_PATH_FORMAT, serviceName);
|
||||
if(zk.getChildren()==null) {
|
||||
return;
|
||||
}
|
||||
List<String> providers = zk.getChildren().forPath(path);
|
||||
for(String provider : providers) {
|
||||
Map<String, String> queryParam = parseQueryString(provider);
|
||||
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
||||
Map<String, String> ipAndPortParam = parseIpAndPortString(path + ZOOKEEPER_SEPARATOR + provider);
|
||||
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
||||
destNamingService.registerInstance(getServiceNameFromCache(serviceName, queryParam),
|
||||
instance);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean delete(TaskDO taskDO) {
|
||||
if (taskDO.getServiceName() == null) {
|
||||
return true;
|
||||
}
|
||||
try {
|
||||
|
||||
CloseableUtils.closeQuietly(treeCacheMap.get(taskDO.getTaskId()));
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
||||
if(!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
||||
if (nacosServiceNameMap.containsKey(taskDO.getServiceName())) {
|
||||
List<Instance> allInstances =
|
||||
destNamingService.getAllInstances(nacosServiceNameMap.get(taskDO.getServiceName()));
|
||||
for (Instance instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||
destNamingService.deregisterInstance(instance.getServiceName(), instance.getIp(),
|
||||
instance.getPort());
|
||||
}
|
||||
nacosServiceNameMap.remove(taskDO.getServiceName());
|
||||
|
||||
}
|
||||
}
|
||||
} else {
|
||||
Set<String> serviceNames = nacosServiceNameMap.keySet();
|
||||
for(String serviceName : serviceNames) {
|
||||
|
||||
if (nacosServiceNameMap.containsKey(serviceName)) {
|
||||
List<Instance> allInstances =
|
||||
destNamingService.getAllInstances(serviceName);
|
||||
for (Instance instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||
destNamingService.deregisterInstance(instance.getServiceName(), instance.getIp(),
|
||||
instance.getPort());
|
||||
}
|
||||
nacosServiceNameMap.remove(serviceName);
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
} catch (Exception e) {
|
||||
log.error("delete task from zookeeper to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
||||
private boolean initializeTreeCache(TaskDO taskDO) {
|
||||
TreeCache treeCache = getTreeCache(taskDO);
|
||||
if (treeCache == null) {
|
||||
logAndRecordSyncError("Failed to obtain TreeCache for taskId: {}", taskDO.getTaskId(), null);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private void logAndRecordSyncError(String message, String taskId, Exception e) {
|
||||
if (e != null) {
|
||||
log.error(message, taskId, e);
|
||||
} else {
|
||||
log.error(message, taskId);
|
||||
}
|
||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||
}
|
||||
|
||||
private void setupListener(TaskDO taskDO, NamingService destNamingService) {
|
||||
TreeCache treeCache = Objects.requireNonNull(getTreeCache(taskDO));
|
||||
treeCache.getListenable().addListener((client, event) -> {
|
||||
try {
|
||||
// INITIALIZED is a special event that is not triggered by the Zookeeper server
|
||||
if(event.getData()==null){
|
||||
log.warn("TreeCache event data is null, taskId:{}", taskDO.getTaskId());
|
||||
return;
|
||||
}
|
||||
String path = event.getData().getPath();
|
||||
Map<String, String> queryParam = parseQueryString(path);
|
||||
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
||||
processEvent(taskDO, destNamingService, event, path, queryParam);
|
||||
}
|
||||
} catch (Exception e) {
|
||||
logAndRecordSyncError("Event process from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
public boolean delete(TaskDO taskDO) {
|
||||
if (taskDO.getServiceName() == null) {
|
||||
return true;
|
||||
}
|
||||
|
||||
CloseableUtils.closeQuietly(treeCacheMap.get(taskDO.getTaskId()));
|
||||
|
||||
try {
|
||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||
if (destNamingService == null) {
|
||||
log.error("Failed to obtain NamingService for destination clusterId: {}", taskDO.getDestClusterId());
|
||||
return false;
|
||||
}
|
||||
|
||||
Set<String> serviceNames = getServiceNamesToDelete(taskDO);
|
||||
deleteInstances(serviceNames, destNamingService, taskDO);
|
||||
|
||||
} catch (Exception e) {
|
||||
log.error("Delete task from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private Set<String> getServiceNamesToDelete(TaskDO taskDO) {
|
||||
if (!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
||||
return new HashSet<>(Collections.singleton(taskDO.getServiceName()));
|
||||
} else {
|
||||
return new HashSet<>(nacosServiceNameMap.keySet());
|
||||
}
|
||||
}
|
||||
|
||||
private void deleteInstances(Set<String> serviceNames, NamingService destNamingService, TaskDO taskDO) {
|
||||
for (String serviceName : serviceNames) {
|
||||
try {
|
||||
List<Instance> allInstances = destNamingService.getAllInstances(serviceName,
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||
for (Instance instance : allInstances) {
|
||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||
destNamingService.deregisterInstance(instance.getServiceName(),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||
}
|
||||
}
|
||||
nacosServiceNameMap.remove(serviceName);
|
||||
} catch (NacosException e) {
|
||||
log.error("Failed to deregister service instance for serviceName: {}", serviceName, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* fetch the Path cache when the task sync
|
||||
*/
|
||||
protected TreeCache getTreeCache(TaskDO taskDO) {
|
||||
return treeCacheMap.computeIfAbsent(taskDO.getTaskId(), (key) -> {
|
||||
try {
|
||||
TreeCache treeCache =
|
||||
new TreeCache(zookeeperServerHolder.get(taskDO.getSourceClusterId(), ""),
|
||||
DUBBO_ROOT_PATH);
|
||||
TreeCache treeCache = new TreeCache(zookeeperServerHolder.get(taskDO.getSourceClusterId()),
|
||||
DUBBO_ROOT_PATH);
|
||||
treeCache.start();
|
||||
return treeCache;
|
||||
} catch (Exception e) {
|
||||
|
@ -252,54 +229,151 @@ public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
|||
return null;
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* The instance information that needs to be synchronized is matched based on the dubbo version and the grouping
|
||||
* name
|
||||
* Checks if the instance information needs to be synchronized based on the dubbo version, grouping name,
|
||||
* and service name.
|
||||
*/
|
||||
protected boolean isMatch(TaskDO taskDO, Map<String, String> queryParam) {
|
||||
Predicate<TaskDO> isVersionEq = (task) -> StringUtils.isBlank(taskDO.getVersion())
|
||||
|| StringUtils.equals(task.getVersion(), queryParam.get(VERSION_KEY));
|
||||
Predicate<TaskDO> isGroupEq = (task) -> StringUtils.isBlank(taskDO.getGroupName())
|
||||
|| StringUtils.equals(task.getGroupName(), queryParam.get(GROUP_KEY));
|
||||
return isVersionEq.and(isGroupEq).test(taskDO);
|
||||
return isVersionMatch(taskDO, queryParam) &&
|
||||
isGroupMatch(taskDO, queryParam) &&
|
||||
isServiceMatch(taskDO, queryParam) ||
|
||||
isMatchAllServices(taskDO);
|
||||
}
|
||||
|
||||
|
||||
private boolean isVersionMatch(TaskDO task, Map<String, String> queryParam) {
|
||||
return StringUtils.isBlank(task.getVersion()) || StringUtils.equals(task.getVersion(), queryParam.get(VERSION_KEY));
|
||||
}
|
||||
|
||||
private boolean isGroupMatch(TaskDO task, Map<String, String> queryParam) {
|
||||
return StringUtils.isBlank(task.getGroupName()) || StringUtils.equals(task.getGroupName(), queryParam.get(GROUP_KEY));
|
||||
}
|
||||
|
||||
private boolean isServiceMatch(TaskDO task, Map<String, String> queryParam) {
|
||||
return StringUtils.isNotBlank(task.getServiceName()) && StringUtils.equals(task.getServiceName(), queryParam.get(INTERFACE_KEY));
|
||||
}
|
||||
|
||||
private boolean isMatchAllServices(TaskDO task) {
|
||||
return StringUtils.isNotBlank(task.getServiceName()) && StringUtils.equals(task.getServiceName(), ALL_SERVICE_NAME_PATTERN);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* create Nacos service instance
|
||||
* Builds a synchronized Nacos instance from Zookeeper data.
|
||||
*
|
||||
* @param queryParam dubbo metadata
|
||||
* @param ipAndPortMap dubbo ip and address
|
||||
* @param queryParam Parameters obtained from the query string.
|
||||
* @param ipAndPortMap IP and port information.
|
||||
* @param taskDO Task details.
|
||||
* @return A fully configured Nacos instance.
|
||||
*/
|
||||
protected Instance buildSyncInstance(Map<String, String> queryParam, Map<String, String> ipAndPortMap,
|
||||
TaskDO taskDO) {
|
||||
Instance temp = new Instance();
|
||||
temp.setIp(ipAndPortMap.get(INSTANCE_IP_KEY));
|
||||
temp.setPort(Integer.parseInt(ipAndPortMap.get(INSTANCE_PORT_KEY)));
|
||||
temp.setServiceName(getServiceNameFromCache(taskDO.getTaskId(), queryParam));
|
||||
temp.setWeight(Double.parseDouble(queryParam.get(WEIGHT_KEY) == null ? "1.0" : queryParam.get(WEIGHT_KEY)));
|
||||
temp.setHealthy(true);
|
||||
|
||||
protected Instance buildSyncInstance(Map<String, String> queryParam, Map<String, String> ipAndPortMap, TaskDO taskDO) {
|
||||
Instance instance = new Instance();
|
||||
instance.setIp(ipAndPortMap.get(INSTANCE_IP_KEY));
|
||||
instance.setPort(Integer.parseInt(ipAndPortMap.get(INSTANCE_PORT_KEY)));
|
||||
instance.setServiceName(getServiceNameFromCache(taskDO.getTaskId(), queryParam));
|
||||
instance.setWeight(parseWeight(queryParam));
|
||||
instance.setHealthy(true);
|
||||
instance.setMetadata(buildMetadata(queryParam, ipAndPortMap, taskDO));
|
||||
return instance;
|
||||
}
|
||||
|
||||
|
||||
private double parseWeight(Map<String, String> queryParam) {
|
||||
try {
|
||||
return Double.parseDouble(queryParam.getOrDefault(WEIGHT_KEY, DEFAULT_WEIGHT));
|
||||
} catch (NumberFormatException e) {
|
||||
log.error("Error parsing weight: {}", queryParam.get(WEIGHT_KEY), e);
|
||||
return Double.parseDouble(DEFAULT_WEIGHT); // Default weight in case of error
|
||||
}
|
||||
}
|
||||
|
||||
private Map<String, String> buildMetadata(Map<String, String> queryParam, Map<String, String> ipAndPortMap, TaskDO taskDO) {
|
||||
Map<String, String> metaData = new HashMap<>(queryParam);
|
||||
metaData.put(PROTOCOL_KEY, ipAndPortMap.get(PROTOCOL_KEY));
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
||||
temp.setMetadata(metaData);
|
||||
return temp;
|
||||
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY, skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||
return metaData;
|
||||
}
|
||||
|
||||
|
||||
private void processEvent(TaskDO taskDO, NamingService destNamingService, TreeCacheEvent event, String path,
|
||||
Map<String, String> queryParam) throws NacosException {
|
||||
if (!com.alibaba.nacossync.util.StringUtils.isDubboProviderPath(path)) {
|
||||
return;
|
||||
}
|
||||
|
||||
Map<String, String> ipAndPortParam = parseIpAndPortString(path);
|
||||
if (ipAndPortParam.isEmpty()) {
|
||||
log.error("Invalid IP and Port data extracted from path: {}", path);
|
||||
return;
|
||||
}
|
||||
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
||||
String serviceName = queryParam.get(INTERFACE_KEY);
|
||||
if (serviceName == null || serviceName.isEmpty()) {
|
||||
log.error("Service name is missing in the query parameters.");
|
||||
return;
|
||||
}
|
||||
switch (event.getType()) {
|
||||
case NODE_ADDED:
|
||||
case NODE_UPDATED:
|
||||
|
||||
destNamingService.registerInstance(getServiceNameFromCache(serviceName, queryParam),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||
break;
|
||||
case NODE_REMOVED:
|
||||
|
||||
destNamingService.deregisterInstance(getServiceNameFromCache(serviceName, queryParam),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), ipAndPortParam.get(INSTANCE_IP_KEY),
|
||||
Integer.parseInt(ipAndPortParam.get(INSTANCE_PORT_KEY)));
|
||||
nacosServiceNameMap.remove(serviceName);
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private void registerAllInstances(TaskDO taskDO, NamingService destNamingService) throws Exception {
|
||||
CuratorFramework zk = zookeeperServerHolder.get(taskDO.getSourceClusterId());
|
||||
if (!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
||||
registerALLInstances0(taskDO, destNamingService, zk, taskDO.getServiceName());
|
||||
} else {
|
||||
// 同步全部
|
||||
List<String> serviceList = zk.getChildren().forPath(DUBBO_ROOT_PATH);
|
||||
for (String serviceName : serviceList) {
|
||||
registerALLInstances0(taskDO, destNamingService, zk, serviceName);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void registerALLInstances0(TaskDO taskDO, NamingService destNamingService, CuratorFramework zk,
|
||||
String serviceName) throws Exception {
|
||||
String path = String.format(DUBBO_PATH_FORMAT, serviceName);
|
||||
if (zk.getChildren() == null) {
|
||||
return;
|
||||
}
|
||||
List<String> providers = zk.getChildren().forPath(path);
|
||||
for (String provider : providers) {
|
||||
Map<String, String> queryParam = parseQueryString(provider);
|
||||
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
||||
Map<String, String> ipAndPortParam = parseIpAndPortString(path + ZOOKEEPER_SEPARATOR + provider);
|
||||
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
||||
destNamingService.registerInstance(getServiceNameFromCache(serviceName, queryParam),
|
||||
getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* cteate Dubbo service name
|
||||
*
|
||||
* @param serviceName dubbo service name
|
||||
* @param queryParam dubbo metadata
|
||||
* @param queryParam dubbo metadata
|
||||
*/
|
||||
protected String getServiceNameFromCache(String serviceName, Map<String, String> queryParam) {
|
||||
return nacosServiceNameMap.computeIfAbsent(serviceName, (key) -> createServiceName(queryParam));
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -23,15 +23,19 @@ import java.util.concurrent.TimeUnit;
|
|||
@Service
|
||||
public class MetricsManager implements CommandLineRunner {
|
||||
|
||||
@Autowired
|
||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
@Autowired
|
||||
private ClusterAccessService clusterAccessService;
|
||||
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
public MetricsManager(SkyWalkerCacheServices skyWalkerCacheServices, ClusterAccessService clusterAccessService,
|
||||
TaskAccessService taskAccessService) {
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
this.taskAccessService = taskAccessService;
|
||||
}
|
||||
|
||||
/**
|
||||
* Callback used to run the bean.
|
||||
*
|
||||
|
|
|
@ -17,17 +17,28 @@
|
|||
package com.alibaba.nacossync.pojo.model;
|
||||
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import lombok.Getter;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.Setter;
|
||||
import lombok.ToString;
|
||||
import org.hibernate.proxy.HibernateProxy;
|
||||
|
||||
import javax.persistence.Entity;
|
||||
import javax.persistence.GeneratedValue;
|
||||
import javax.persistence.GenerationType;
|
||||
import javax.persistence.Id;
|
||||
import javax.persistence.Table;
|
||||
import java.io.Serializable;
|
||||
|
||||
import javax.persistence.*;
|
||||
|
||||
import lombok.Data;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: EnvDO.java, v 0.1 2018-09-25 PM 4:17 NacosSync Exp $$
|
||||
*/
|
||||
@Data
|
||||
@Getter
|
||||
@Setter
|
||||
@ToString
|
||||
@RequiredArgsConstructor
|
||||
@Entity
|
||||
@Table(name = "cluster")
|
||||
public class ClusterDO implements Serializable {
|
||||
|
@ -66,4 +77,34 @@ public class ClusterDO implements Serializable {
|
|||
*/
|
||||
private String password;
|
||||
|
||||
private String namespace;
|
||||
|
||||
private Integer clusterLevel;
|
||||
|
||||
@Override
|
||||
public final boolean equals(Object o) {
|
||||
if (this == o) {
|
||||
return true;
|
||||
}
|
||||
if (o == null) {
|
||||
return false;
|
||||
}
|
||||
Class<?> oEffectiveClass =
|
||||
o instanceof HibernateProxy ? ((HibernateProxy) o).getHibernateLazyInitializer().getPersistentClass()
|
||||
: o.getClass();
|
||||
Class<?> thisEffectiveClass =
|
||||
this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||
.getPersistentClass() : this.getClass();
|
||||
if (thisEffectiveClass != oEffectiveClass) {
|
||||
return false;
|
||||
}
|
||||
ClusterDO clusterDO = (ClusterDO) o;
|
||||
return getId() != null && Objects.equals(getId(), clusterDO.getId());
|
||||
}
|
||||
|
||||
@Override
|
||||
public final int hashCode() {
|
||||
return this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||
.getPersistentClass().hashCode() : getClass().hashCode();
|
||||
}
|
||||
}
|
||||
|
|
|
@ -16,15 +16,28 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.pojo.model;
|
||||
|
||||
import javax.persistence.*;
|
||||
|
||||
import lombok.Data;
|
||||
import lombok.Getter;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.Setter;
|
||||
import lombok.ToString;
|
||||
import org.hibernate.proxy.HibernateProxy;
|
||||
|
||||
import javax.persistence.Entity;
|
||||
import javax.persistence.GeneratedValue;
|
||||
import javax.persistence.GenerationType;
|
||||
import javax.persistence.Id;
|
||||
import javax.persistence.Table;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: SystemConfig.java, v 0.1 2018-09-26 上午1:48 NacosSync Exp $$
|
||||
*/
|
||||
@Data
|
||||
@Getter
|
||||
@Setter
|
||||
@ToString
|
||||
@RequiredArgsConstructor
|
||||
@Entity
|
||||
@Table(name = "system_config")
|
||||
public class SystemConfigDO {
|
||||
|
@ -34,5 +47,31 @@ public class SystemConfigDO {
|
|||
private String configKey;
|
||||
private String configValue;
|
||||
private String configDesc;
|
||||
|
||||
|
||||
@Override
|
||||
public final boolean equals(Object o) {
|
||||
if (this == o) {
|
||||
return true;
|
||||
}
|
||||
if (o == null) {
|
||||
return false;
|
||||
}
|
||||
Class<?> oEffectiveClass =
|
||||
o instanceof HibernateProxy ? ((HibernateProxy) o).getHibernateLazyInitializer().getPersistentClass()
|
||||
: o.getClass();
|
||||
Class<?> thisEffectiveClass =
|
||||
this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||
.getPersistentClass() : this.getClass();
|
||||
if (thisEffectiveClass != oEffectiveClass) {
|
||||
return false;
|
||||
}
|
||||
SystemConfigDO that = (SystemConfigDO) o;
|
||||
return getId() != null && Objects.equals(getId(), that.getId());
|
||||
}
|
||||
|
||||
@Override
|
||||
public final int hashCode() {
|
||||
return this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||
.getPersistentClass().hashCode() : getClass().hashCode();
|
||||
}
|
||||
}
|
||||
|
|
|
@ -12,16 +12,29 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.pojo.model;
|
||||
|
||||
import lombok.Data;
|
||||
|
||||
import javax.persistence.*;
|
||||
import lombok.Getter;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.Setter;
|
||||
import lombok.ToString;
|
||||
import org.hibernate.proxy.HibernateProxy;
|
||||
|
||||
import javax.persistence.Entity;
|
||||
import javax.persistence.GeneratedValue;
|
||||
import javax.persistence.GenerationType;
|
||||
import javax.persistence.Id;
|
||||
import javax.persistence.Table;
|
||||
import java.io.Serializable;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: TaskDo.java, v 0.1 2018-09-24 PM11:53 NacosSync Exp $$
|
||||
*/
|
||||
@Data
|
||||
@Getter
|
||||
@Setter
|
||||
@ToString
|
||||
@RequiredArgsConstructor
|
||||
@Entity
|
||||
@Table(name = "task")
|
||||
public class TaskDO implements Serializable {
|
||||
|
@ -70,4 +83,31 @@ public class TaskDO implements Serializable {
|
|||
* operation id,The operation id follow when the task status changes
|
||||
*/
|
||||
private String operationId;
|
||||
|
||||
@Override
|
||||
public final boolean equals(Object o) {
|
||||
if (this == o) {
|
||||
return true;
|
||||
}
|
||||
if (o == null) {
|
||||
return false;
|
||||
}
|
||||
Class<?> oEffectiveClass =
|
||||
o instanceof HibernateProxy ? ((HibernateProxy) o).getHibernateLazyInitializer().getPersistentClass()
|
||||
: o.getClass();
|
||||
Class<?> thisEffectiveClass =
|
||||
this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||
.getPersistentClass() : this.getClass();
|
||||
if (thisEffectiveClass != oEffectiveClass) {
|
||||
return false;
|
||||
}
|
||||
TaskDO taskDO = (TaskDO) o;
|
||||
return getId() != null && Objects.equals(getId(), taskDO.getId());
|
||||
}
|
||||
|
||||
@Override
|
||||
public final int hashCode() {
|
||||
return this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||
.getPersistentClass().hashCode() : getClass().hashCode();
|
||||
}
|
||||
}
|
||||
|
|
|
@ -17,10 +17,10 @@
|
|||
package com.alibaba.nacossync.pojo.request;
|
||||
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import java.util.List;
|
||||
|
||||
import lombok.Data;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: AddClusterRequest.java, v 0.1 2018-09-25 PM 10:27 NacosSync Exp $$
|
||||
|
@ -52,5 +52,6 @@ public class ClusterAddRequest extends BaseRequest {
|
|||
* The password of the Nacos.
|
||||
*/
|
||||
private String password;
|
||||
private String namespace;
|
||||
|
||||
}
|
||||
|
|
|
@ -0,0 +1,44 @@
|
|||
/*
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.pojo.request;
|
||||
|
||||
import lombok.Data;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: TaskAddAllRequest.java, v 0.1 2022-03-23 AM12:13 NacosSync Exp $$
|
||||
*/
|
||||
@Data
|
||||
public class TaskAddAllRequest extends BaseRequest {
|
||||
|
||||
/**
|
||||
* eg: b7bacb110199d5bb83b9757038fadeb0 .
|
||||
*/
|
||||
private String sourceClusterId;
|
||||
|
||||
/**
|
||||
* eg: bbdad57833a0e4f0981f6f3349005617 .
|
||||
*/
|
||||
private String destClusterId;
|
||||
|
||||
/**
|
||||
* whether to exclude subscriber.
|
||||
*/
|
||||
private boolean excludeConsumer = true;
|
||||
|
||||
}
|
|
@ -14,33 +14,53 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.pojo.view;
|
||||
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import java.io.Serializable;
|
||||
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import lombok.Data;
|
||||
|
||||
import java.io.Serializable;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: ClusterModel.java, v 0.1 2018-09-25 下午11:09 NacosSync Exp $$
|
||||
*/
|
||||
@Data
|
||||
public class ClusterModel implements Serializable {
|
||||
|
||||
|
||||
private String clusterId;
|
||||
|
||||
/**
|
||||
* json format,["192.168.1:8080","192.168.2?key=1"]
|
||||
*/
|
||||
private String connectKeyList;
|
||||
|
||||
/**
|
||||
* cluster name, eg:cluster of ShangHai(edas-sh)
|
||||
*/
|
||||
private String clusterName;
|
||||
|
||||
/**
|
||||
* cluster type, eg cluster of CS,cluster of Nacos,
|
||||
*
|
||||
* @see ClusterTypeEnum
|
||||
*/
|
||||
private String clusterType;
|
||||
|
||||
private String namespace;
|
||||
|
||||
private String userName;
|
||||
|
||||
public static ClusterModel from(ClusterDO clusterDO) {
|
||||
ClusterModel clusterModel = new ClusterModel();
|
||||
clusterModel.setClusterId(clusterDO.getClusterId());
|
||||
clusterModel.setConnectKeyList(clusterDO.getConnectKeyList());
|
||||
clusterModel.setClusterType(clusterDO.getClusterType());
|
||||
clusterModel.setClusterName(clusterDO.getClusterName());
|
||||
clusterModel.setNamespace(clusterDO.getNamespace());
|
||||
clusterModel.setUserName(clusterDO.getUserName());
|
||||
return clusterModel;
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,8 +14,10 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.pojo.view;
|
||||
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import lombok.Data;
|
||||
|
||||
/**
|
||||
|
@ -24,12 +26,27 @@ import lombok.Data;
|
|||
*/
|
||||
@Data
|
||||
public class TaskModel {
|
||||
|
||||
|
||||
private String taskId;
|
||||
|
||||
private String sourceClusterId;
|
||||
|
||||
private String destClusterId;
|
||||
|
||||
private String serviceName;
|
||||
|
||||
private String groupName;
|
||||
|
||||
private String taskStatus;
|
||||
|
||||
|
||||
public static TaskModel from(TaskDO taskDO) {
|
||||
TaskModel taskModel = new TaskModel();
|
||||
taskModel.setDestClusterId(taskDO.getDestClusterId());
|
||||
taskModel.setGroupName(taskDO.getGroupName());
|
||||
taskModel.setServiceName(taskDO.getServiceName());
|
||||
taskModel.setSourceClusterId(taskDO.getSourceClusterId());
|
||||
taskModel.setTaskStatus(taskDO.getTaskStatus());
|
||||
taskModel.setTaskId(taskDO.getTaskId());
|
||||
return taskModel;
|
||||
}
|
||||
}
|
||||
|
|
|
@ -45,8 +45,7 @@ public class SkyWalkerTemplate {
|
|||
}
|
||||
|
||||
private static <T extends BaseResult> void initExceptionResult(T result, Throwable e) {
|
||||
if (e instanceof SkyWalkerException) {
|
||||
SkyWalkerException skyWalkerException = (SkyWalkerException) e;
|
||||
if (e instanceof SkyWalkerException skyWalkerException) {
|
||||
if (null != skyWalkerException.getResultCode()) {
|
||||
result.setResultCode(skyWalkerException.getResultCode().getCode());
|
||||
}
|
||||
|
|
|
@ -14,21 +14,20 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import com.alibaba.nacossync.pojo.request.ClusterAddRequest;
|
||||
import com.alibaba.nacossync.pojo.result.ClusterAddResult;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
/**
|
||||
|
@ -38,47 +37,53 @@ import org.springframework.stereotype.Service;
|
|||
@Slf4j
|
||||
@Service
|
||||
public class ClusterAddProcessor implements Processor<ClusterAddRequest, ClusterAddResult> {
|
||||
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
|
||||
@Autowired
|
||||
private ClusterAccessService clusterAccessService;
|
||||
|
||||
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public ClusterAddProcessor(ClusterAccessService clusterAccessService, ObjectMapper objectMapper) {
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
this.objectMapper = objectMapper;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(ClusterAddRequest clusterAddRequest, ClusterAddResult clusterAddResult,
|
||||
Object... others) throws Exception {
|
||||
public void process(ClusterAddRequest clusterAddRequest, ClusterAddResult clusterAddResult, Object... others)
|
||||
throws Exception {
|
||||
ClusterDO clusterDO = new ClusterDO();
|
||||
|
||||
if (null == clusterAddRequest.getConnectKeyList() || 0 == clusterAddRequest.getConnectKeyList().size()) {
|
||||
|
||||
|
||||
if (null == clusterAddRequest.getConnectKeyList() || clusterAddRequest.getConnectKeyList().isEmpty()) {
|
||||
|
||||
throw new SkyWalkerException("集群列表不能为空!");
|
||||
}
|
||||
|
||||
if (StringUtils.isBlank(clusterAddRequest.getClusterName()) || StringUtils
|
||||
.isBlank(clusterAddRequest.getClusterType())) {
|
||||
|
||||
|
||||
if (StringUtils.isBlank(clusterAddRequest.getClusterName()) || StringUtils.isBlank(
|
||||
clusterAddRequest.getClusterType())) {
|
||||
|
||||
throw new SkyWalkerException("集群名字或者类型不能为空!");
|
||||
}
|
||||
|
||||
|
||||
if (!ClusterTypeEnum.contains(clusterAddRequest.getClusterType())) {
|
||||
|
||||
|
||||
throw new SkyWalkerException("集群类型不存在:" + clusterAddRequest.getClusterType());
|
||||
}
|
||||
|
||||
|
||||
String clusterId = SkyWalkerUtil.generateClusterId(clusterAddRequest);
|
||||
|
||||
|
||||
if (null != clusterAccessService.findByClusterId(clusterId)) {
|
||||
|
||||
|
||||
throw new SkyWalkerException("重复插入,clusterId已存在:" + clusterId);
|
||||
}
|
||||
|
||||
|
||||
clusterDO.setClusterId(clusterId);
|
||||
clusterDO.setClusterName(clusterAddRequest.getClusterName());
|
||||
clusterDO.setClusterType(clusterAddRequest.getClusterType());
|
||||
clusterDO.setConnectKeyList(JSONObject.toJSONString(clusterAddRequest.getConnectKeyList()));
|
||||
clusterDO.setConnectKeyList(objectMapper.writeValueAsString(clusterAddRequest.getConnectKeyList()));
|
||||
clusterDO.setUserName(clusterAddRequest.getUserName());
|
||||
clusterDO.setPassword(clusterAddRequest.getPassword());
|
||||
clusterDO.setNamespace(clusterAddRequest.getNamespace());
|
||||
clusterDO.setClusterLevel(0);
|
||||
clusterAccessService.insert(clusterDO);
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,15 +14,16 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.pojo.result.ClusterDeleteResult;
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||
import com.alibaba.nacossync.pojo.request.ClusterDeleteRequest;
|
||||
import com.alibaba.nacossync.pojo.result.ClusterDeleteResult;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
|
@ -30,15 +31,26 @@ import com.alibaba.nacossync.template.Processor;
|
|||
*/
|
||||
@Service
|
||||
public class ClusterDeleteProcessor implements Processor<ClusterDeleteRequest, ClusterDeleteResult> {
|
||||
|
||||
@Autowired
|
||||
private ClusterAccessService clusterAccessService;
|
||||
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
public ClusterDeleteProcessor(ClusterAccessService clusterAccessService, TaskAccessService taskAccessService) {
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
this.taskAccessService = taskAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(ClusterDeleteRequest clusterDeleteRequest,
|
||||
ClusterDeleteResult clusterDeleteResult, Object... others) throws Exception {
|
||||
|
||||
public void process(ClusterDeleteRequest clusterDeleteRequest, ClusterDeleteResult clusterDeleteResult,
|
||||
Object... others) throws Exception {
|
||||
int count = taskAccessService.countByDestClusterIdOrSourceClusterId(clusterDeleteRequest.getClusterId(),
|
||||
clusterDeleteRequest.getClusterId());
|
||||
if (count > 0) {
|
||||
throw new SkyWalkerException(String.format("集群下有%d个任务,请先删除任务", count));
|
||||
}
|
||||
|
||||
clusterAccessService.deleteByClusterId(clusterDeleteRequest.getClusterId());
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,44 +14,37 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.pojo.result.ClusterDetailQueryResult;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import com.alibaba.nacossync.pojo.request.ClusterDetailQueryRequest;
|
||||
import com.alibaba.nacossync.pojo.result.ClusterDetailQueryResult;
|
||||
import com.alibaba.nacossync.pojo.view.ClusterModel;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: ClusterDetailQueryProcessor.java, v 0.1 2018-09-30 PM2:39 NacosSync Exp $$
|
||||
*/
|
||||
@Service
|
||||
public class ClusterDetailQueryProcessor
|
||||
implements
|
||||
Processor<ClusterDetailQueryRequest, ClusterDetailQueryResult> {
|
||||
@Autowired
|
||||
private ClusterAccessService clusterAccessService;
|
||||
|
||||
public class ClusterDetailQueryProcessor implements Processor<ClusterDetailQueryRequest, ClusterDetailQueryResult> {
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
public ClusterDetailQueryProcessor(ClusterAccessService clusterAccessService) {
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(ClusterDetailQueryRequest clusterDetailQueryRequest,
|
||||
ClusterDetailQueryResult clusterDetailQueryResult, Object... others)
|
||||
throws Exception {
|
||||
|
||||
ClusterDO clusterDO = clusterAccessService.findByClusterId(clusterDetailQueryRequest
|
||||
.getClusterId());
|
||||
|
||||
ClusterModel clusterModel = new ClusterModel();
|
||||
clusterModel.setClusterId(clusterDO.getClusterId());
|
||||
clusterModel.setConnectKeyList(clusterDO.getConnectKeyList());
|
||||
clusterModel.setClusterType(clusterDO.getClusterType());
|
||||
clusterModel.setClusterName(clusterDO.getClusterName());
|
||||
|
||||
clusterDetailQueryResult.setClusterModel(clusterModel);
|
||||
|
||||
ClusterDetailQueryResult clusterDetailQueryResult, Object... others) throws Exception {
|
||||
|
||||
ClusterDO clusterDO = clusterAccessService.findByClusterId(clusterDetailQueryRequest.getClusterId());
|
||||
|
||||
clusterDetailQueryResult.setClusterModel(ClusterModel.from(clusterDO));
|
||||
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,70 +14,60 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import com.alibaba.nacossync.pojo.request.ClusterListQueryRequest;
|
||||
import com.alibaba.nacossync.pojo.result.ClusterListQueryResult;
|
||||
import com.alibaba.nacossync.pojo.view.ClusterModel;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.data.domain.Page;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.pojo.result.ClusterListQueryResult;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import com.alibaba.nacossync.pojo.request.ClusterListQueryRequest;
|
||||
import com.alibaba.nacossync.pojo.view.ClusterModel;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: ClusterListQueryProcessor.java, v 0.1 2018-09-30 PM2:33 NacosSync Exp $$
|
||||
*/
|
||||
@Service
|
||||
public class ClusterListQueryProcessor implements
|
||||
Processor<ClusterListQueryRequest, ClusterListQueryResult> {
|
||||
|
||||
@Autowired
|
||||
private ClusterAccessService clusterAccessService;
|
||||
|
||||
public class ClusterListQueryProcessor implements Processor<ClusterListQueryRequest, ClusterListQueryResult> {
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
public ClusterListQueryProcessor(ClusterAccessService clusterAccessService) {
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(ClusterListQueryRequest clusterListQueryRequest,
|
||||
ClusterListQueryResult clusterListQueryResult, Object... others) {
|
||||
|
||||
public void process(ClusterListQueryRequest clusterListQueryRequest, ClusterListQueryResult clusterListQueryResult,
|
||||
Object... others) {
|
||||
|
||||
Page<ClusterDO> clusterDOS;
|
||||
|
||||
|
||||
if (StringUtils.isNotBlank(clusterListQueryRequest.getClusterName())) {
|
||||
|
||||
|
||||
QueryCondition queryCondition = new QueryCondition();
|
||||
queryCondition.setServiceName(clusterListQueryRequest.getClusterName());
|
||||
clusterDOS = clusterAccessService.findPageCriteria(clusterListQueryRequest.getPageNum() - 1,
|
||||
clusterListQueryRequest.getPageSize(), queryCondition);
|
||||
|
||||
|
||||
} else {
|
||||
|
||||
|
||||
clusterDOS = clusterAccessService.findPageNoCriteria(clusterListQueryRequest.getPageNum() - 1,
|
||||
clusterListQueryRequest.getPageSize());
|
||||
|
||||
|
||||
}
|
||||
|
||||
List<ClusterModel> clusterModels = new ArrayList<>();
|
||||
clusterDOS.forEach(clusterDO -> {
|
||||
|
||||
ClusterModel clusterModel = new ClusterModel();
|
||||
clusterModel.setClusterId(clusterDO.getClusterId());
|
||||
clusterModel.setClusterName(clusterDO.getClusterName());
|
||||
clusterModel.setClusterType(clusterDO.getClusterType());
|
||||
clusterModel.setConnectKeyList(clusterDO.getConnectKeyList());
|
||||
clusterModels.add(clusterModel);
|
||||
});
|
||||
|
||||
List<ClusterModel> clusterModels = clusterDOS.stream().map(ClusterModel::from).toList();
|
||||
clusterListQueryResult.setClusterModels(clusterModels);
|
||||
clusterListQueryResult.setTotalPage(clusterDOS.getTotalPages());
|
||||
clusterListQueryResult.setCurrentSize(clusterModels.size());
|
||||
clusterListQueryResult.setTotalSize(clusterDOS.getTotalElements());
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
|
|
|
@ -0,0 +1,250 @@
|
|||
/*
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import com.alibaba.nacos.api.exception.NacosException;
|
||||
import com.alibaba.nacos.api.naming.CommonParams;
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
import com.alibaba.nacos.client.naming.NacosNamingService;
|
||||
import com.alibaba.nacos.client.naming.net.NamingProxy;
|
||||
import com.alibaba.nacos.client.naming.utils.UtilAndComs;
|
||||
import com.alibaba.nacos.common.utils.HttpMethod;
|
||||
import com.alibaba.nacos.common.utils.JacksonUtils;
|
||||
import com.alibaba.nacos.common.utils.StringUtils;
|
||||
import com.alibaba.nacossync.constant.TaskStatusEnum;
|
||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||
import com.alibaba.nacossync.extension.SyncManagerService;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.pojo.request.TaskAddAllRequest;
|
||||
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
||||
import com.alibaba.nacossync.pojo.result.TaskAddResult;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||
import lombok.Data;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.util.ReflectionUtils;
|
||||
|
||||
import javax.annotation.Nullable;
|
||||
import java.lang.reflect.Field;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
import static com.alibaba.nacossync.constant.SkyWalkerConstants.GROUP_NAME_PARAM;
|
||||
import static com.alibaba.nacossync.constant.SkyWalkerConstants.PAGE_NO;
|
||||
import static com.alibaba.nacossync.constant.SkyWalkerConstants.PAGE_SIZE;
|
||||
import static com.alibaba.nacossync.constant.SkyWalkerConstants.SERVICE_NAME_PARAM;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: TaskAddAllProcessor.java, v 0.1 2022-03-23 PM11:40 NacosSync Exp $$
|
||||
*/
|
||||
@Slf4j
|
||||
@Service
|
||||
public class TaskAddAllProcessor implements Processor<TaskAddAllRequest, TaskAddResult> {
|
||||
|
||||
private static final String CONSUMER_PREFIX = "consumers:";
|
||||
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
|
||||
private final SyncManagerService syncManagerService;
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
private final ClusterAccessService clusterAccessService;
|
||||
|
||||
public TaskAddAllProcessor(NacosServerHolder nacosServerHolder, SyncManagerService syncManagerService,
|
||||
TaskAccessService taskAccessService, ClusterAccessService clusterAccessService) {
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
this.syncManagerService = syncManagerService;
|
||||
this.taskAccessService = taskAccessService;
|
||||
this.clusterAccessService = clusterAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(TaskAddAllRequest addAllRequest, TaskAddResult taskAddResult, Object... others)
|
||||
throws Exception {
|
||||
|
||||
ClusterDO destCluster = clusterAccessService.findByClusterId(addAllRequest.getDestClusterId());
|
||||
|
||||
ClusterDO sourceCluster = clusterAccessService.findByClusterId(addAllRequest.getSourceClusterId());
|
||||
|
||||
if (Objects.isNull(destCluster) || Objects.isNull(sourceCluster)) {
|
||||
throw new SkyWalkerException("Please check if the source or target cluster exists.");
|
||||
}
|
||||
|
||||
if (Objects.isNull(syncManagerService.getSyncService(sourceCluster.getClusterId(), destCluster.getClusterId()))) {
|
||||
throw new SkyWalkerException("current sync type not supported.");
|
||||
}
|
||||
// TODO 目前仅支持 Nacos 为源的同步类型,待完善更多类型支持。
|
||||
final NamingService sourceNamingService = nacosServerHolder.get(sourceCluster.getClusterId());
|
||||
if (sourceNamingService == null) {
|
||||
throw new SkyWalkerException("only support sync type that the source of the Nacos.");
|
||||
}
|
||||
|
||||
final EnhanceNamingService enhanceNamingService = new EnhanceNamingService(sourceNamingService);
|
||||
final CatalogServiceResult catalogServiceResult = enhanceNamingService.catalogServices(null, null);
|
||||
if (catalogServiceResult == null || catalogServiceResult.getCount() <= 0) {
|
||||
throw new SkyWalkerException("sourceCluster data empty");
|
||||
}
|
||||
|
||||
for (ServiceView serviceView : catalogServiceResult.getServiceList()) {
|
||||
// exclude subscriber
|
||||
if (addAllRequest.isExcludeConsumer() && serviceView.getName().startsWith(CONSUMER_PREFIX)) {
|
||||
continue;
|
||||
}
|
||||
TaskAddRequest taskAddRequest = new TaskAddRequest();
|
||||
taskAddRequest.setSourceClusterId(sourceCluster.getClusterId());
|
||||
taskAddRequest.setDestClusterId(destCluster.getClusterId());
|
||||
taskAddRequest.setServiceName(serviceView.getName());
|
||||
taskAddRequest.setGroupName(serviceView.getGroupName());
|
||||
this.dealTask(addAllRequest, taskAddRequest);
|
||||
}
|
||||
}
|
||||
|
||||
private void dealTask(TaskAddAllRequest addAllRequest, TaskAddRequest taskAddRequest) throws Exception {
|
||||
|
||||
String taskId = SkyWalkerUtil.generateTaskId(taskAddRequest);
|
||||
TaskDO taskDO = taskAccessService.findByTaskId(taskId);
|
||||
if (null == taskDO) {
|
||||
taskDO = new TaskDO();
|
||||
taskDO.setTaskId(taskId);
|
||||
taskDO.setDestClusterId(addAllRequest.getDestClusterId());
|
||||
taskDO.setSourceClusterId(addAllRequest.getSourceClusterId());
|
||||
taskDO.setServiceName(taskAddRequest.getServiceName());
|
||||
taskDO.setVersion(taskAddRequest.getVersion());
|
||||
taskDO.setGroupName(taskAddRequest.getGroupName());
|
||||
taskDO.setNameSpace(taskAddRequest.getNameSpace());
|
||||
taskDO.setTaskStatus(TaskStatusEnum.SYNC.getCode());
|
||||
taskDO.setWorkerIp(SkyWalkerUtil.getLocalIp());
|
||||
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
||||
|
||||
} else {
|
||||
taskDO.setTaskStatus(TaskStatusEnum.SYNC.getCode());
|
||||
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
||||
}
|
||||
taskAccessService.addTask(taskDO);
|
||||
}
|
||||
|
||||
static class EnhanceNamingService {
|
||||
|
||||
protected NamingService delegate;
|
||||
|
||||
protected NamingProxy serverProxy;
|
||||
|
||||
protected EnhanceNamingService(NamingService namingService) {
|
||||
if (!(namingService instanceof NacosNamingService)) {
|
||||
throw new IllegalArgumentException(
|
||||
"namingService only support instance of com.alibaba.nacos.client.naming.NacosNamingService.");
|
||||
}
|
||||
this.delegate = namingService;
|
||||
|
||||
// serverProxy
|
||||
final Field serverProxyField = ReflectionUtils.findField(NacosNamingService.class, "serverProxy");
|
||||
assert serverProxyField != null;
|
||||
ReflectionUtils.makeAccessible(serverProxyField);
|
||||
this.serverProxy = (NamingProxy) ReflectionUtils.getField(serverProxyField, delegate);
|
||||
}
|
||||
|
||||
public CatalogServiceResult catalogServices(@Nullable String serviceName, @Nullable String group)
|
||||
throws NacosException {
|
||||
int pageNo = 1; // start with 1
|
||||
int pageSize = 100;
|
||||
|
||||
final CatalogServiceResult result = catalogServices(serviceName, group, pageNo, pageSize);
|
||||
|
||||
CatalogServiceResult tmpResult = result;
|
||||
|
||||
while (Objects.nonNull(tmpResult) && tmpResult.serviceList.size() >= pageSize) {
|
||||
pageNo++;
|
||||
tmpResult = catalogServices(serviceName, group, pageNo, pageSize);
|
||||
|
||||
if (tmpResult != null) {
|
||||
result.serviceList.addAll(tmpResult.serviceList);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* @see com.alibaba.nacos.client.naming.core.HostReactor#getServiceInfoDirectlyFromServer(String, String)
|
||||
*/
|
||||
public CatalogServiceResult catalogServices(@Nullable String serviceName, @Nullable String group, int pageNo,
|
||||
int pageSize) throws NacosException {
|
||||
|
||||
// pageNo
|
||||
// pageSize
|
||||
// serviceNameParam
|
||||
// groupNameParam
|
||||
final Map<String, String> params = new HashMap<>(8);
|
||||
params.put(CommonParams.NAMESPACE_ID, serverProxy.getNamespaceId());
|
||||
params.put(SERVICE_NAME_PARAM, serviceName);
|
||||
params.put(GROUP_NAME_PARAM, group);
|
||||
params.put(PAGE_NO, String.valueOf(pageNo));
|
||||
params.put(PAGE_SIZE, String.valueOf(pageSize));
|
||||
|
||||
final String result = this.serverProxy.reqApi(UtilAndComs.nacosUrlBase + "/catalog/services", params,
|
||||
HttpMethod.GET);
|
||||
if (StringUtils.isNotEmpty(result)) {
|
||||
return JacksonUtils.toObj(result, CatalogServiceResult.class);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy from Nacos Server.
|
||||
*/
|
||||
@Data
|
||||
static class ServiceView {
|
||||
|
||||
private String name;
|
||||
|
||||
private String groupName;
|
||||
|
||||
private int clusterCount;
|
||||
|
||||
private int ipCount;
|
||||
|
||||
private int healthyInstanceCount;
|
||||
|
||||
private String triggerFlag;
|
||||
|
||||
}
|
||||
|
||||
@Data
|
||||
static class CatalogServiceResult {
|
||||
|
||||
/**
|
||||
* count,not equal serviceList.size .
|
||||
*/
|
||||
private int count;
|
||||
|
||||
private List<ServiceView> serviceList;
|
||||
|
||||
}
|
||||
|
||||
}
|
|
@ -89,7 +89,7 @@ public class TaskAddProcessor implements Processor<TaskAddRequest, TaskAddResult
|
|||
taskDO.setTaskStatus(TaskStatusEnum.SYNC.getCode());
|
||||
taskDO.setWorkerIp(SkyWalkerUtil.getLocalIp());
|
||||
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
||||
|
||||
|
||||
} else {
|
||||
|
||||
taskDO.setTaskStatus(TaskStatusEnum.SYNC.getCode());
|
||||
|
|
|
@ -16,19 +16,12 @@
|
|||
*/
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.pojo.request.TaskDeleteInBatchRequest;
|
||||
import com.alibaba.nacossync.pojo.result.BaseResult;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
|
||||
/**
|
||||
|
@ -40,20 +33,15 @@ import lombok.extern.slf4j.Slf4j;
|
|||
@Service
|
||||
public class TaskDeleteInBatchProcessor implements Processor<TaskDeleteInBatchRequest, BaseResult> {
|
||||
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
public TaskDeleteInBatchProcessor(TaskAccessService taskAccessService) {
|
||||
this.taskAccessService = taskAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(TaskDeleteInBatchRequest taskBatchDeleteRequest, BaseResult baseResult,
|
||||
Object... others) {
|
||||
//
|
||||
// String[] taskIds= taskBatchDeleteRequest.getTaskIds();
|
||||
// List<TaskDO> taskDOs = new ArrayList<TaskDO>();
|
||||
// for (String taskId : taskIds) {
|
||||
// TaskDO taskDO = new TaskDO();
|
||||
// taskDO.setTaskId(taskId);
|
||||
// taskDOs.add(taskDO);
|
||||
// }
|
||||
taskAccessService.deleteTaskInBatch(taskBatchDeleteRequest.getTaskIds());
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,9 +14,12 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.event.DeleteAllSubTaskEvent;
|
||||
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.pojo.request.TaskDeleteRequest;
|
||||
|
@ -24,7 +27,6 @@ import com.alibaba.nacossync.pojo.result.BaseResult;
|
|||
import com.alibaba.nacossync.template.Processor;
|
||||
import com.google.common.eventbus.EventBus;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
/**
|
||||
|
@ -34,18 +36,28 @@ import org.springframework.stereotype.Service;
|
|||
@Slf4j
|
||||
@Service
|
||||
public class TaskDeleteProcessor implements Processor<TaskDeleteRequest, BaseResult> {
|
||||
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
@Autowired
|
||||
private EventBus eventBus;
|
||||
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
private final EventBus eventBus;
|
||||
|
||||
public TaskDeleteProcessor(TaskAccessService taskAccessService, EventBus eventBus) {
|
||||
this.taskAccessService = taskAccessService;
|
||||
this.eventBus = eventBus;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(TaskDeleteRequest taskDeleteRequest, BaseResult baseResult,
|
||||
Object... others) {
|
||||
public void process(TaskDeleteRequest taskDeleteRequest, BaseResult baseResult, Object... others) {
|
||||
TaskDO taskDO = taskAccessService.findByTaskId(taskDeleteRequest.getTaskId());
|
||||
eventBus.post(new DeleteTaskEvent(taskDO));
|
||||
log.info("删除同步任务数据之前,发出一个同步事件:" + taskDO);
|
||||
// delete all sub task when ServiceName is all
|
||||
if (SkyWalkerConstants.NACOS_ALL_SERVICE_NAME.equalsIgnoreCase(taskDO.getServiceName())) {
|
||||
eventBus.post(new DeleteAllSubTaskEvent(taskDO));
|
||||
} else {
|
||||
eventBus.post(new DeleteTaskEvent(taskDO));
|
||||
}
|
||||
log.info("删除同步任务数据之前,发出一个同步事件:{}", taskDO);
|
||||
taskAccessService.deleteTaskById(taskDeleteRequest.getTaskId());
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -14,17 +14,17 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||
import com.alibaba.nacossync.pojo.result.TaskDetailQueryResult;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.pojo.request.TaskDetailQueryRequest;
|
||||
import com.alibaba.nacossync.pojo.result.TaskDetailQueryResult;
|
||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
/**
|
||||
|
@ -34,29 +34,23 @@ import org.springframework.stereotype.Service;
|
|||
@Slf4j
|
||||
@Service
|
||||
public class TaskDetailProcessor implements Processor<TaskDetailQueryRequest, TaskDetailQueryResult> {
|
||||
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
public TaskDetailProcessor(TaskAccessService taskAccessService) {
|
||||
this.taskAccessService = taskAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(TaskDetailQueryRequest taskDetailQueryRequest, TaskDetailQueryResult taskDetailQueryResult, Object... others)
|
||||
throws Exception {
|
||||
|
||||
public void process(TaskDetailQueryRequest taskDetailQueryRequest, TaskDetailQueryResult taskDetailQueryResult,
|
||||
Object... others) throws Exception {
|
||||
|
||||
TaskDO taskDO = taskAccessService.findByTaskId(taskDetailQueryRequest.getTaskId());
|
||||
|
||||
|
||||
if (null == taskDO) {
|
||||
throw new SkyWalkerException("taskDo is null,taskId :" + taskDetailQueryRequest.getTaskId());
|
||||
}
|
||||
|
||||
TaskModel taskModel = new TaskModel();
|
||||
|
||||
taskModel.setDestClusterId(taskDO.getDestClusterId());
|
||||
taskModel.setGroupName(taskDO.getGroupName());
|
||||
taskModel.setServiceName(taskDO.getServiceName());
|
||||
taskModel.setSourceClusterId(taskDO.getSourceClusterId());
|
||||
taskModel.setTaskStatus(taskDO.getTaskStatus());
|
||||
taskModel.setTaskId(taskDO.getTaskId());
|
||||
|
||||
taskDetailQueryResult.setTaskModel(taskModel);
|
||||
|
||||
taskDetailQueryResult.setTaskModel(TaskModel.from(taskDO));
|
||||
}
|
||||
}
|
||||
|
|
|
@ -14,25 +14,23 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.pojo.request.TaskListQueryRequest;
|
||||
import com.alibaba.nacossync.pojo.result.TaskListQueryResult;
|
||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.data.domain.Page;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.pojo.result.TaskListQueryResult;
|
||||
import com.alibaba.nacossync.pojo.request.TaskListQueryRequest;
|
||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
|
@ -41,42 +39,34 @@ import com.alibaba.nacossync.template.Processor;
|
|||
@Service
|
||||
@Slf4j
|
||||
public class TaskListQueryProcessor implements Processor<TaskListQueryRequest, TaskListQueryResult> {
|
||||
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
public TaskListQueryProcessor(TaskAccessService taskAccessService) {
|
||||
this.taskAccessService = taskAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(TaskListQueryRequest taskListQueryRequest,
|
||||
TaskListQueryResult taskListQueryResult, Object... others) {
|
||||
|
||||
public void process(TaskListQueryRequest taskListQueryRequest, TaskListQueryResult taskListQueryResult,
|
||||
Object... others) {
|
||||
|
||||
Page<TaskDO> taskDOPage;
|
||||
|
||||
|
||||
if (StringUtils.isNotBlank(taskListQueryRequest.getServiceName())) {
|
||||
|
||||
|
||||
QueryCondition queryCondition = new QueryCondition();
|
||||
queryCondition.setServiceName(taskListQueryRequest.getServiceName());
|
||||
taskDOPage = taskAccessService.findPageCriteria(taskListQueryRequest.getPageNum() - 1,
|
||||
taskListQueryRequest.getPageSize(), queryCondition);
|
||||
} else {
|
||||
|
||||
|
||||
taskDOPage = taskAccessService.findPageNoCriteria(taskListQueryRequest.getPageNum() - 1,
|
||||
taskListQueryRequest.getPageSize());
|
||||
|
||||
|
||||
}
|
||||
|
||||
List<TaskModel> taskList = new ArrayList<>();
|
||||
|
||||
taskDOPage.forEach(taskDO -> {
|
||||
TaskModel taskModel = new TaskModel();
|
||||
taskModel.setTaskId(taskDO.getTaskId());
|
||||
taskModel.setDestClusterId(taskDO.getDestClusterId());
|
||||
taskModel.setSourceClusterId(taskDO.getSourceClusterId());
|
||||
taskModel.setServiceName(taskDO.getServiceName());
|
||||
taskModel.setGroupName(taskDO.getGroupName());
|
||||
taskModel.setTaskStatus(taskDO.getTaskStatus());
|
||||
taskList.add(taskModel);
|
||||
});
|
||||
|
||||
|
||||
List<TaskModel> taskList = taskDOPage.stream().map(TaskModel::from).toList();
|
||||
|
||||
taskListQueryResult.setTaskModels(taskList);
|
||||
taskListQueryResult.setTotalPage(taskDOPage.getTotalPages());
|
||||
taskListQueryResult.setTotalSize(taskDOPage.getTotalElements());
|
||||
|
|
|
@ -14,21 +14,19 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.template.processor;
|
||||
|
||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.constant.TaskStatusEnum;
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||
import com.alibaba.nacossync.pojo.result.BaseResult;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.pojo.request.TaskUpdateRequest;
|
||||
import com.alibaba.nacossync.pojo.result.BaseResult;
|
||||
import com.alibaba.nacossync.template.Processor;
|
||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
|
@ -37,29 +35,33 @@ import com.alibaba.nacossync.template.Processor;
|
|||
@Slf4j
|
||||
@Service
|
||||
public class TaskUpdateProcessor implements Processor<TaskUpdateRequest, BaseResult> {
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
public TaskUpdateProcessor(TaskAccessService taskAccessService) {
|
||||
this.taskAccessService = taskAccessService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void process(TaskUpdateRequest taskUpdateRequest, BaseResult baseResult,
|
||||
Object... others) throws Exception {
|
||||
|
||||
public void process(TaskUpdateRequest taskUpdateRequest, BaseResult baseResult, Object... others) throws Exception {
|
||||
|
||||
TaskDO taskDO = taskAccessService.findByTaskId(taskUpdateRequest.getTaskId());
|
||||
|
||||
|
||||
if (!TaskStatusEnum.contains(taskUpdateRequest.getTaskStatus())) {
|
||||
throw new SkyWalkerException(
|
||||
"taskUpdateRequest.getTaskStatus() is not exist , value is :"
|
||||
+ taskUpdateRequest.getTaskStatus());
|
||||
"taskUpdateRequest.getTaskStatus() is not exist , value is :" + taskUpdateRequest.getTaskStatus());
|
||||
}
|
||||
|
||||
|
||||
if (null == taskDO) {
|
||||
throw new SkyWalkerException("taskDo is null ,taskId is :"
|
||||
+ taskUpdateRequest.getTaskId());
|
||||
throw new SkyWalkerException("taskDo is null ,taskId is :" + taskUpdateRequest.getTaskId());
|
||||
}
|
||||
|
||||
|
||||
taskDO.setTaskStatus(taskUpdateRequest.getTaskStatus());
|
||||
|
||||
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
||||
|
||||
|
||||
taskAccessService.addTask(taskDO);
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
|
|
@ -0,0 +1,252 @@
|
|||
/*
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.timer;
|
||||
|
||||
import com.alibaba.nacos.api.naming.NamingService;
|
||||
import com.alibaba.nacos.api.naming.pojo.ListView;
|
||||
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
|
||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||
import com.alibaba.nacossync.constant.TaskStatusEnum;
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.event.DeleteAllSubTaskEvent;
|
||||
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||
import com.alibaba.nacossync.event.SyncTaskEvent;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||
import com.google.common.eventbus.EventBus;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.BeanUtils;
|
||||
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* when the database task service name is empty, check all the services in the cluster and create a synchronization
|
||||
* task.
|
||||
*/
|
||||
@Slf4j
|
||||
public class CheckRunningStatusAllNacosThread implements Runnable {
|
||||
|
||||
private final Map<String, Set<String>> subTaskService = new ConcurrentHashMap<>();
|
||||
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
|
||||
private final EventBus eventBus;
|
||||
|
||||
|
||||
public CheckRunningStatusAllNacosThread(MetricsManager metricsManager, TaskAccessService taskAccessService,
|
||||
NacosServerHolder nacosServerHolder, EventBus eventBus) {
|
||||
this.metricsManager = metricsManager;
|
||||
this.taskAccessService = taskAccessService;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
this.eventBus = eventBus;
|
||||
}
|
||||
|
||||
/**
|
||||
* Synchronize data based on the ns level.
|
||||
*/
|
||||
@Override
|
||||
public void run() {
|
||||
|
||||
try {
|
||||
List<TaskDO> tasks = taskAccessService.findAllByServiceNameEqualAll();
|
||||
if (CollectionUtils.isEmpty(tasks)) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Get the set of all task IDs
|
||||
Set<String> taskIdSet = tasks.stream().map(TaskDO::getTaskId).collect(Collectors.toSet());
|
||||
|
||||
// Filter and handle sub-tasks that need to be deleted. This handles the case where tasks have been deleted
|
||||
// but sub-tasks still exist.
|
||||
subTaskService.entrySet().stream()
|
||||
.filter(entry -> shouldDeleteSubTasks(entry, taskIdSet))
|
||||
.forEach(entry -> postDeleteAllSubTaskEvent(entry.getKey()));
|
||||
|
||||
// Handle regular tasks
|
||||
tasks.forEach(this::processTask);
|
||||
} catch (Exception e) {
|
||||
log.warn("CheckRunningStatusThread Exception", e);
|
||||
metricsManager.recordError(MetricsStatisticsType.DISPATCHER_TASK);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Listens for the event of deleting all sub-tasks and handles the delete operation.
|
||||
*
|
||||
* @param deleteAllSubTaskEvent The event object containing the task information to be deleted.
|
||||
*/
|
||||
public void listenerDeleteAllTaskEvent(DeleteAllSubTaskEvent deleteAllSubTaskEvent) {
|
||||
// Retrieve the task object
|
||||
TaskDO task = deleteAllSubTaskEvent.getTaskDO();
|
||||
|
||||
// Retrieve the task ID
|
||||
String taskId = task.getTaskId();
|
||||
|
||||
// Remove the set of service names corresponding to the task ID from subTaskService
|
||||
Set<String> serviceNameSet = subTaskService.remove(taskId);
|
||||
|
||||
// If the set of service names is empty, return immediately
|
||||
if (CollectionUtils.isEmpty(serviceNameSet)) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Build the list of sub-tasks pending removal
|
||||
List<TaskDO> servicesPendingRemoval = serviceNameSet.stream()
|
||||
.map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||
.toList();
|
||||
|
||||
// Handle the removal of the pending sub-tasks
|
||||
handleRemoval(servicesPendingRemoval, serviceNameSet);
|
||||
}
|
||||
|
||||
|
||||
private boolean shouldDeleteSubTasks(Map.Entry<String, Set<String>> entry, Set<String> taskIdSet) {
|
||||
return !taskIdSet.contains(entry.getKey()) && !entry.getValue().isEmpty();
|
||||
}
|
||||
|
||||
private void postDeleteAllSubTaskEvent(String taskId) {
|
||||
TaskDO taskDO = new TaskDO();
|
||||
taskDO.setTaskId(taskId);
|
||||
eventBus.post(new DeleteAllSubTaskEvent(taskDO));
|
||||
}
|
||||
|
||||
/**
|
||||
* Processes the given task by determining the services that need to be inserted and removed,
|
||||
* and performs the corresponding operations.
|
||||
*
|
||||
* @param task The task object to be processed.
|
||||
*/
|
||||
private void processTask(TaskDO task) {
|
||||
// Retrieve the set of services for the task, creating a new set if it does not exist
|
||||
Set<String> serviceSet = subTaskService.computeIfAbsent(task.getTaskId(), k -> ConcurrentHashMap.newKeySet());
|
||||
|
||||
// Get the list of all service names associated with the task
|
||||
List<String> serviceNameList = getAllServiceName(task);
|
||||
|
||||
// Determine the services that need to be inserted (those not in the current service set)
|
||||
List<TaskDO> servicesPendingInsertion = serviceNameList.stream()
|
||||
.filter(serviceName -> !serviceSet.contains(serviceName))
|
||||
.map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||
.toList();
|
||||
|
||||
// Determine the services that need to be removed (those in the current service set but not in the service name list)
|
||||
List<TaskDO> servicesPendingRemoval = serviceSet.stream()
|
||||
.filter(serviceName -> !serviceNameList.contains(serviceName))
|
||||
.map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||
.toList();
|
||||
|
||||
|
||||
// If all lists are empty, there is nothing to process
|
||||
if (CollectionUtils.isEmpty(serviceNameList) && CollectionUtils.isEmpty(servicesPendingInsertion)
|
||||
&& CollectionUtils.isEmpty(servicesPendingRemoval)) {
|
||||
log.debug("No service found for task: {}", task.getTaskId());
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
// If the task status is SYNC, handle the insertion of services
|
||||
if (TaskStatusEnum.SYNC.getCode().equals(task.getTaskStatus())) {
|
||||
handleInsertion(servicesPendingInsertion, serviceSet);
|
||||
}
|
||||
// Handle the removal of services
|
||||
handleRemoval(servicesPendingRemoval, serviceSet);
|
||||
|
||||
if (TaskStatusEnum.DELETE.getCode().equals(task.getTaskStatus())) {
|
||||
List<TaskDO> allSubTasks = serviceNameList.stream().map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||
.collect(Collectors.toList());
|
||||
handleRemoval(allSubTasks, serviceSet);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handles the insertion of services.
|
||||
*
|
||||
* @param servicesPendingInsertion The list of services to be inserted.
|
||||
* @param serviceSet The set of services.
|
||||
*/
|
||||
private void handleInsertion(List<TaskDO> servicesPendingInsertion, Set<String> serviceSet) {
|
||||
BatchTaskExecutor.batchOperation(servicesPendingInsertion, t -> {
|
||||
eventBus.post(new SyncTaskEvent(t));
|
||||
serviceSet.add(t.getServiceName());
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Handles the removal of services.
|
||||
*
|
||||
* @param servicesPendingRemoval The list of services to be removed.
|
||||
* @param serviceSet The set of services.
|
||||
*/
|
||||
private void handleRemoval(List<TaskDO> servicesPendingRemoval, Set<String> serviceSet) {
|
||||
BatchTaskExecutor.batchOperation(servicesPendingRemoval, t -> {
|
||||
eventBus.post(new DeleteTaskEvent(t));
|
||||
serviceSet.remove(t.getServiceName());
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Builds a sub-task object for the given task and service name.
|
||||
*
|
||||
* @param serviceName The service name.
|
||||
* @return The constructed sub-task object.
|
||||
*/
|
||||
private static TaskDO buildSubTaskDO(TaskDO taskDO, String serviceName) {
|
||||
TaskDO task = new TaskDO();
|
||||
|
||||
BeanUtils.copyProperties(taskDO, task);
|
||||
task.setTaskId(SkyWalkerUtil.generateTaskId(serviceName, taskDO.getGroupName(), taskDO.getSourceClusterId(),
|
||||
taskDO.getDestClusterId()));
|
||||
task.setServiceName(serviceName);
|
||||
return task;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieves all service names associated with the given task.
|
||||
*
|
||||
* @return A list of service names.
|
||||
*/
|
||||
private List<String> getAllServiceName(TaskDO taskDO) {
|
||||
NamingService namingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||
if (namingService == null) {
|
||||
log.warn("naming service is null or not found, clusterId:{}", taskDO.getSourceClusterId());
|
||||
return Collections.emptyList();
|
||||
}
|
||||
try {
|
||||
ListView<String> servicesOfServer = namingService.getServicesOfServer(0, Integer.MAX_VALUE,
|
||||
taskDO.getGroupName());
|
||||
return servicesOfServer.getData();
|
||||
} catch (Exception e) {
|
||||
log.error("query service list failure", e);
|
||||
}
|
||||
|
||||
return Collections.emptyList();
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
|
@ -14,24 +14,23 @@
|
|||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.timer;
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.CommandLineRunner;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.pojo.FinishedTask;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.boot.CommandLineRunner;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.stream.StreamSupport;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
|
@ -40,57 +39,53 @@ import com.alibaba.nacossync.pojo.model.TaskDO;
|
|||
@Slf4j
|
||||
@Service
|
||||
public class CleanExceedOperationIdTimer implements CommandLineRunner {
|
||||
|
||||
@Autowired
|
||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
|
||||
@Autowired
|
||||
private ScheduledExecutorService scheduledExecutorService;
|
||||
|
||||
|
||||
private static final long INITIAL_DELAY = 0;
|
||||
|
||||
private static final long PERIOD = 12;
|
||||
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
private final ScheduledExecutorService scheduledExecutorService;
|
||||
|
||||
public CleanExceedOperationIdTimer(SkyWalkerCacheServices skyWalkerCacheServices,
|
||||
TaskAccessService taskAccessService, ScheduledExecutorService scheduledExecutorService) {
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.taskAccessService = taskAccessService;
|
||||
this.scheduledExecutorService = scheduledExecutorService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run(String... args) {
|
||||
/** Clean up the OperationId cache once every 12 hours */
|
||||
scheduledExecutorService.scheduleWithFixedDelay(new CleanExceedOperationIdThread(), 0, 12,
|
||||
TimeUnit.HOURS);
|
||||
|
||||
scheduledExecutorService.scheduleWithFixedDelay(new CleanExceedOperationIdThread(), INITIAL_DELAY, PERIOD,
|
||||
TimeUnit.HOURS);
|
||||
log.info("CleanExceedOperationIdTimer has started successfully");
|
||||
|
||||
}
|
||||
|
||||
|
||||
private class CleanExceedOperationIdThread implements Runnable {
|
||||
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
|
||||
|
||||
try {
|
||||
|
||||
Map<String, FinishedTask> finishedTaskMap = skyWalkerCacheServices
|
||||
.getFinishedTaskMap();
|
||||
|
||||
Iterable<TaskDO> taskDOS = taskAccessService.findAll();
|
||||
|
||||
Set<String> operationIds = getDbOperations(taskDOS);
|
||||
|
||||
for (String operationId : finishedTaskMap.keySet()) {
|
||||
|
||||
if (!operationIds.contains(operationId)) {
|
||||
|
||||
finishedTaskMap.remove(operationId);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Map<String, FinishedTask> finishedTaskMap = skyWalkerCacheServices.getFinishedTaskMap();
|
||||
Set<String> operationIds = getDbOperations(taskAccessService.findAll());
|
||||
finishedTaskMap.keySet().removeIf(operationId -> !operationIds.contains(operationId));
|
||||
|
||||
} catch (Exception e) {
|
||||
log.warn("CleanExceedOperationIdThread Exception", e);
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
||||
|
||||
private Set<String> getDbOperations(Iterable<TaskDO> taskDOS) {
|
||||
Set<String> operationIds = new HashSet<>();
|
||||
|
||||
taskDOS.forEach(taskDO -> operationIds.add(taskDO.getOperationId()));
|
||||
return operationIds;
|
||||
return StreamSupport.stream(taskDOS.spliterator(), false).map(TaskDO::getOperationId)
|
||||
.collect(Collectors.toSet());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
@ -22,14 +22,15 @@ import com.alibaba.nacossync.constant.TaskStatusEnum;
|
|||
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||
import com.alibaba.nacossync.event.SyncTaskEvent;
|
||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.google.common.eventbus.EventBus;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.CommandLineRunner;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
|
@ -40,27 +41,44 @@ import java.util.concurrent.TimeUnit;
|
|||
@Slf4j
|
||||
@Service
|
||||
public class QuerySyncTaskTimer implements CommandLineRunner {
|
||||
@Autowired
|
||||
private MetricsManager metricsManager;
|
||||
|
||||
private static final int INITIAL_DELAY = 0;
|
||||
|
||||
private static final int DELAY = 3000;
|
||||
|
||||
private final MetricsManager metricsManager;
|
||||
|
||||
@Autowired
|
||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||
|
||||
@Autowired
|
||||
private TaskAccessService taskAccessService;
|
||||
private final TaskAccessService taskAccessService;
|
||||
|
||||
@Autowired
|
||||
private EventBus eventBus;
|
||||
|
||||
@Autowired
|
||||
private ScheduledExecutorService scheduledExecutorService;
|
||||
private final EventBus eventBus;
|
||||
|
||||
private final ScheduledExecutorService scheduledExecutorService;
|
||||
|
||||
private final NacosServerHolder nacosServerHolder;
|
||||
|
||||
public QuerySyncTaskTimer(MetricsManager metricsManager, SkyWalkerCacheServices skyWalkerCacheServices,
|
||||
TaskAccessService taskAccessService, EventBus eventBus, ScheduledExecutorService scheduledExecutorService,
|
||||
NacosServerHolder nacosServerHolder) {
|
||||
this.metricsManager = metricsManager;
|
||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||
this.taskAccessService = taskAccessService;
|
||||
this.eventBus = eventBus;
|
||||
this.scheduledExecutorService = scheduledExecutorService;
|
||||
this.nacosServerHolder = nacosServerHolder;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run(String... args) {
|
||||
/** Fetch the task list from the database every 3 seconds */
|
||||
scheduledExecutorService.scheduleWithFixedDelay(new CheckRunningStatusThread(), 0, 3000,
|
||||
scheduledExecutorService.scheduleWithFixedDelay(new CheckRunningStatusThread(), INITIAL_DELAY, DELAY,
|
||||
TimeUnit.MILLISECONDS);
|
||||
|
||||
|
||||
scheduledExecutorService.scheduleWithFixedDelay(new CheckRunningStatusAllNacosThread(metricsManager,
|
||||
taskAccessService, nacosServerHolder, eventBus), INITIAL_DELAY, DELAY,
|
||||
TimeUnit.MILLISECONDS);
|
||||
log.info("QuerySyncTaskTimer has started successfully");
|
||||
}
|
||||
|
||||
private class CheckRunningStatusThread implements Runnable {
|
||||
|
@ -68,10 +86,10 @@ public class QuerySyncTaskTimer implements CommandLineRunner {
|
|||
@Override
|
||||
public void run() {
|
||||
|
||||
Long start = System.currentTimeMillis();
|
||||
long start = System.currentTimeMillis();
|
||||
try {
|
||||
|
||||
Iterable<TaskDO> taskDOS = taskAccessService.findAll();
|
||||
List<TaskDO> taskDOS = taskAccessService.findAllByServiceNameNotEqualAll();
|
||||
|
||||
taskDOS.forEach(taskDO -> {
|
||||
|
||||
|
@ -83,13 +101,13 @@ public class QuerySyncTaskTimer implements CommandLineRunner {
|
|||
if (TaskStatusEnum.SYNC.getCode().equals(taskDO.getTaskStatus())) {
|
||||
|
||||
eventBus.post(new SyncTaskEvent(taskDO));
|
||||
log.info("从数据库中查询到一个同步任务,发出一个同步事件:" + taskDO);
|
||||
log.info("从数据库中查询到一个同步任务,发出一个同步事件:{}", taskDO);
|
||||
}
|
||||
|
||||
if (TaskStatusEnum.DELETE.getCode().equals(taskDO.getTaskStatus())) {
|
||||
|
||||
eventBus.post(new DeleteTaskEvent(taskDO));
|
||||
log.info("从数据库中查询到一个删除任务,发出一个同步事件:" + taskDO);
|
||||
log.info("从数据库中查询到一个删除任务,发出一个同步事件:{}", taskDO);
|
||||
}
|
||||
});
|
||||
|
||||
|
|
|
@ -17,7 +17,6 @@ import com.alibaba.nacossync.extension.event.SpecialSyncEvent;
|
|||
import com.alibaba.nacossync.extension.event.SpecialSyncEventBus;
|
||||
import com.google.common.eventbus.EventBus;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.CommandLineRunner;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
|
@ -33,20 +32,25 @@ import java.util.concurrent.TimeUnit;
|
|||
@Service
|
||||
public class SpecialSyncEventTimer implements CommandLineRunner {
|
||||
|
||||
@Autowired
|
||||
private SpecialSyncEventBus specialSyncEventBus;
|
||||
private final SpecialSyncEventBus specialSyncEventBus;
|
||||
|
||||
@Autowired
|
||||
private EventBus eventBus;
|
||||
|
||||
@Autowired
|
||||
private ScheduledExecutorService scheduledExecutorService;
|
||||
private final EventBus eventBus;
|
||||
|
||||
private final ScheduledExecutorService scheduledExecutorService;
|
||||
|
||||
public SpecialSyncEventTimer(SpecialSyncEventBus specialSyncEventBus, EventBus eventBus,
|
||||
ScheduledExecutorService scheduledExecutorService) {
|
||||
this.specialSyncEventBus = specialSyncEventBus;
|
||||
this.eventBus = eventBus;
|
||||
this.scheduledExecutorService = scheduledExecutorService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run(String... args) throws Exception {
|
||||
|
||||
scheduledExecutorService.scheduleWithFixedDelay(new SpecialSyncEventTimer.SpecialSyncEventThread(), 0, 3000,
|
||||
TimeUnit.MILLISECONDS);
|
||||
log.info("SpecialSyncEventTimer has started successfully");
|
||||
}
|
||||
|
||||
private class SpecialSyncEventThread implements Runnable {
|
||||
|
@ -58,9 +62,9 @@ public class SpecialSyncEventTimer implements CommandLineRunner {
|
|||
allSpecialSyncEvent.stream()
|
||||
.filter(specialSyncEvent -> TaskStatusEnum.SYNC.getCode()
|
||||
.equals(specialSyncEvent.getTaskDO().getTaskStatus()))
|
||||
.forEach(specialSyncEvent -> eventBus.post(specialSyncEvent));
|
||||
.forEach(eventBus::post);
|
||||
} catch (Exception e) {
|
||||
log.warn("SpecialSyncEventThread Exception", e);
|
||||
log.error("Exception occurred while processing special sync events", e);
|
||||
|
||||
}
|
||||
}
|
||||
|
|
|
@ -0,0 +1,104 @@
|
|||
package com.alibaba.nacossync.util;
|
||||
|
||||
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
|
||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||
import com.google.common.base.Stopwatch;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.*;
|
||||
import java.util.function.Consumer;
|
||||
|
||||
@Slf4j
|
||||
public class BatchTaskExecutor {
|
||||
|
||||
private static final int MAX_THREAD_NUM = 200;
|
||||
private static final ExecutorService executorService = Executors.newFixedThreadPool(MAX_THREAD_NUM);
|
||||
|
||||
/**
|
||||
* Batch operation method
|
||||
*
|
||||
* @param items Task list
|
||||
* @param operation Operation to be executed
|
||||
*/
|
||||
public static void batchOperation(List<TaskDO> items, Consumer<TaskDO> operation) {
|
||||
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||
|
||||
List<Tuple<Integer, List<TaskDO>>> taskGroupList = averageAssign(items, MAX_THREAD_NUM);
|
||||
|
||||
// Create a CompletableFuture for each task group
|
||||
CompletableFuture<?>[] futures = taskGroupList.stream().map(tuple -> CompletableFuture.runAsync(() -> {
|
||||
for (TaskDO taskDO : tuple.getT2()) {
|
||||
try {
|
||||
// Add timeout control for each task to avoid long-running tasks
|
||||
CompletableFuture.runAsync(() -> operation.accept(taskDO), executorService)
|
||||
.orTimeout(5, TimeUnit.SECONDS) // Task timeout set to 5 seconds
|
||||
.exceptionally(ex -> {
|
||||
log.error("Task execution timed out: {}", taskDO.getServiceName(), ex);
|
||||
return null;
|
||||
}).join();
|
||||
} catch (Exception e) {
|
||||
log.error("Error occurred during task execution: {}", taskDO.getServiceName(), e);
|
||||
}
|
||||
}
|
||||
}, executorService)).toArray(CompletableFuture[]::new);
|
||||
|
||||
try {
|
||||
// Wait for all tasks to complete
|
||||
CompletableFuture.allOf(futures).join();
|
||||
} catch (Exception e) {
|
||||
log.error("Error occurred during sync operation", e);
|
||||
} finally {
|
||||
log.info("Total sync tasks: {}, Execution time: {} ms", items.size(), stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Divide a list into n sublists, mainly implemented by offset
|
||||
* @param source collection to be divided
|
||||
* @param limit maximum value
|
||||
* @return list after division
|
||||
* @param <T> object type
|
||||
*/
|
||||
private static <T> List<Tuple<Integer, List<T>>> averageAssign(List<T> source, int limit) {
|
||||
if (CollectionUtils.isEmpty(source)) {
|
||||
return Collections.emptyList();
|
||||
}
|
||||
|
||||
int size = source.size();
|
||||
int listCount = (int) Math.ceil((double) size / limit); // Calculate the number of sublists
|
||||
int remainder = size % listCount; // Calculate the number of remaining elements after even distribution
|
||||
List<Tuple<Integer, List<T>>> result = new ArrayList<>(listCount); // Initialize the result list with the expected size
|
||||
|
||||
for (int i = 0, assigned = 0; i < listCount; i++) {
|
||||
int sublistSize = size / listCount + (remainder-- > 0 ? 1 : 0); // Determine the size of each sublist, distribute remaining elements
|
||||
List<T> sublist = new ArrayList<>(source.subList(assigned, assigned + sublistSize)); // Create the sublist
|
||||
result.add(Tuple.of(i, sublist)); // Add the sublist to the result
|
||||
assigned += sublistSize; // Update the assigned index
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Shutdown the executor service to avoid resource leakage
|
||||
*/
|
||||
public static void shutdown() {
|
||||
executorService.shutdown();
|
||||
try {
|
||||
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||
executorService.shutdownNow();
|
||||
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||
log.error("Executor service did not terminate");
|
||||
}
|
||||
}
|
||||
} catch (InterruptedException ie) {
|
||||
executorService.shutdownNow();
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
package com.alibaba.nacossync.util;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collection;
|
||||
import java.util.Iterator;
|
||||
|
||||
public abstract class Collections {
|
||||
|
||||
public static Collection subtract(final Collection a, final Collection b) {
|
||||
ArrayList list = new ArrayList( a );
|
||||
for (Iterator it = b.iterator(); it.hasNext();) {
|
||||
list.remove(it.next());
|
||||
}
|
||||
return list;
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
|
@ -1,15 +1,3 @@
|
|||
/*
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE
|
||||
* file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
|
||||
* to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the
|
||||
* License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package com.alibaba.nacossync.util;
|
||||
|
||||
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
|
||||
|
@ -20,16 +8,21 @@ import java.util.Map;
|
|||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* @author paderlol
|
||||
* @date: 2019-04-25 00:01
|
||||
* Utility class for handling Consul metadata.
|
||||
*/
|
||||
public class ConsulUtils {
|
||||
public static Map<String, String> transferMetadata(List<String> tags) {
|
||||
Map<String, String> metadata = new HashMap<>();
|
||||
if (!CollectionUtils.isEmpty(tags)) {
|
||||
return tags.stream().filter(tag -> tag.split("=", -1).length == 2).map(tag -> tag.split("=", -1))
|
||||
.collect(Collectors.toMap(tagSplitArray -> tagSplitArray[0], tagSplitArray -> tagSplitArray[1]));
|
||||
if (CollectionUtils.isEmpty(tags)) {
|
||||
return new HashMap<>();
|
||||
}
|
||||
return metadata;
|
||||
|
||||
return tags.stream()
|
||||
.map(tag -> tag.split("=", -1))
|
||||
.filter(tagArray -> tagArray.length == 2)
|
||||
.collect(Collectors.toMap(
|
||||
tagArray -> tagArray[0],
|
||||
tagArray -> tagArray[1],
|
||||
(existing, replacement) -> existing // In case of duplicate keys, keep the existing value
|
||||
));
|
||||
}
|
||||
}
|
||||
|
|
|
@ -20,7 +20,6 @@ import org.apache.commons.lang3.StringUtils;
|
|||
import java.math.BigDecimal;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.function.Predicate;
|
||||
|
||||
/**
|
||||
* @author paderlol
|
||||
|
@ -49,45 +48,51 @@ public final class DubboConstants {
|
|||
|
||||
public static final String DUBBO_ROOT_PATH = "/dubbo";
|
||||
public static final String ALL_SERVICE_NAME_PATTERN = "*";
|
||||
|
||||
|
||||
/**
|
||||
* if Dubbo version greater than 2.7.2, service name is providers:interface:version:
|
||||
* if Dubbo version less than 2.7.2, service name is providers:interface:version
|
||||
* @param queryParam
|
||||
* @return
|
||||
* Creates a service name based on Dubbo version compatibility.
|
||||
* if Dubbo version greater than 2.7.2, service name is providers:interface:version:
|
||||
* if Dubbo version less than 2.7.2, service name is providers:interface:version
|
||||
*
|
||||
* @param queryParam the query parameters that include keys such as interface, version, group, etc.
|
||||
* @return the constructed service name string
|
||||
*/
|
||||
public static String createServiceName(Map<String, String> queryParam) {
|
||||
|
||||
String group = queryParam.get(GROUP_KEY);
|
||||
String release = queryParam.get(RELEASE_KEY);
|
||||
Predicate<String> isBlankGroup = StringUtils::isBlank;
|
||||
Predicate<String> isNotBlankRelease = StringUtils::isNotBlank;
|
||||
String serviceName = Joiner.on(SEPARATOR_KEY).skipNulls().join(CATALOG_KEY, queryParam.get(INTERFACE_KEY),
|
||||
queryParam.get(VERSION_KEY), group);
|
||||
|
||||
//TODO The code here is to deal with service metadata format problems caused by dubbo version incompatibility
|
||||
if (isBlankGroup.test(group) && isNotBlankRelease.test(release)) {
|
||||
|
||||
String baseServiceName = Joiner.on(SEPARATOR_KEY).skipNulls().join(CATALOG_KEY, queryParam.get(INTERFACE_KEY),
|
||||
queryParam.get(VERSION_KEY), group);
|
||||
|
||||
if (StringUtils.isBlank(group) && StringUtils.isNotBlank(release)) {
|
||||
List<String> versions = Splitter.on(RELEASE_SEPARATOR_KEY).splitToList(release);
|
||||
if (!CollectionUtils.isEmpty(versions) && versions.size() >= DUBBO_VERSION_INDEX) {
|
||||
String firstVersion = versions.get(0);
|
||||
String secondVersion = versions.get(1);
|
||||
if (DUBBO_VERSION_INDEX == Integer.parseInt(firstVersion)) {
|
||||
if (MIDDLE_DUBBO_VERSION_INDEX <= versions.size()) {
|
||||
String thirdVersion = versions.get(2);
|
||||
BigDecimal bigDecimal =
|
||||
new BigDecimal(Joiner.on(RELEASE_SEPARATOR_KEY).join(secondVersion, thirdVersion));
|
||||
if (bigDecimal.compareTo(COMPARE_NUMBER) > 0) {
|
||||
serviceName = serviceName.concat(SEPARATOR_KEY);
|
||||
}
|
||||
} else if (versions.size() == DUBBO_VERSION_INDEX && Integer.parseInt(secondVersion) > 7) {
|
||||
serviceName = serviceName.concat(SEPARATOR_KEY);
|
||||
}
|
||||
} else if (MIN_DUBBO_VERSION < Integer.parseInt(firstVersion)) {
|
||||
serviceName = serviceName.concat(SEPARATOR_KEY);
|
||||
BigDecimal bigDecimal = new BigDecimal(Joiner.on(RELEASE_SEPARATOR_KEY).join(secondVersion,
|
||||
versions.size() > 2 ? versions.get(2) : "0"));
|
||||
if (isVersionRequiresSeparator(firstVersion, secondVersion, bigDecimal)) {
|
||||
baseServiceName = baseServiceName.concat(SEPARATOR_KEY);
|
||||
}
|
||||
}
|
||||
}
|
||||
return serviceName;
|
||||
return baseServiceName;
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if the version requires a separator to be appended to the service name.
|
||||
*
|
||||
* @param firstVersion the major version
|
||||
* @param secondVersion the minor version
|
||||
* @param bigDecimal the version number as BigDecimal
|
||||
* @return true if separator should be added, otherwise false
|
||||
*/
|
||||
private static boolean isVersionRequiresSeparator(String firstVersion, String secondVersion, BigDecimal bigDecimal) {
|
||||
int majorVersion = Integer.parseInt(firstVersion);
|
||||
int minorVersion = Integer.parseInt(secondVersion);
|
||||
|
||||
return (DUBBO_VERSION_INDEX == majorVersion && (MIDDLE_DUBBO_VERSION_INDEX <= minorVersion ||
|
||||
bigDecimal.compareTo(COMPARE_NUMBER) > 0)) || (MIN_DUBBO_VERSION < majorVersion);
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
@ -0,0 +1,11 @@
|
|||
package com.alibaba.nacossync.util;
|
||||
|
||||
import com.alibaba.nacos.api.common.Constants;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
|
||||
public class NacosUtils {
|
||||
|
||||
public static String getGroupNameOrDefault(String groupName) {
|
||||
return StringUtils.defaultIfBlank(groupName, Constants.DEFAULT_GROUP);
|
||||
}
|
||||
}
|
|
@ -1,19 +1,3 @@
|
|||
/*
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.alibaba.nacossync.util;
|
||||
|
||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||
|
@ -22,135 +6,139 @@ import com.alibaba.nacossync.pojo.model.TaskDO;
|
|||
import com.alibaba.nacossync.pojo.request.ClusterAddRequest;
|
||||
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
||||
import com.google.common.base.Joiner;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
|
||||
import java.io.UnsupportedEncodingException;
|
||||
import java.net.Inet4Address;
|
||||
import java.net.InetAddress;
|
||||
import java.net.NetworkInterface;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.util.Enumeration;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* @author NacosSync
|
||||
* @version $Id: SkyWalkerUtil.java, v 0.1 2018-09-26 AM12:10 NacosSync Exp $$
|
||||
*/
|
||||
* Utility class for various operations in SkyWalker.
|
||||
*/
|
||||
public class SkyWalkerUtil {
|
||||
|
||||
|
||||
private static final String SEPARATOR = ":";
|
||||
private static final String MD5_ALGORITHM = "MD5";
|
||||
|
||||
/**
|
||||
* Generates an MD5 hash for the given string.
|
||||
*
|
||||
* Gets the string md5
|
||||
* @param value
|
||||
* @return
|
||||
* @param value The string to be encrypted.
|
||||
* @return The encrypted string, or an empty string if encryption fails.
|
||||
*/
|
||||
public static String StringToMd5(String value) {
|
||||
{
|
||||
try {
|
||||
MessageDigest md5 = MessageDigest.getInstance("MD5");
|
||||
md5.update(value.getBytes("UTF-8"));
|
||||
byte[] encryption = md5.digest();
|
||||
StringBuffer strBuf = new StringBuffer();
|
||||
for (int i = 0; i < encryption.length; i++) {
|
||||
if (Integer.toHexString(0xff & encryption[i]).length() == 1) {
|
||||
strBuf.append("0").append(Integer.toHexString(0xff & encryption[i]));
|
||||
} else {
|
||||
strBuf.append(Integer.toHexString(0xff & encryption[i]));
|
||||
}
|
||||
}
|
||||
return strBuf.toString();
|
||||
} catch (NoSuchAlgorithmException e) {
|
||||
return "";
|
||||
} catch (UnsupportedEncodingException e) {
|
||||
return "";
|
||||
public static String stringToMd5(String value) {
|
||||
if (StringUtils.isBlank(value)) {
|
||||
return "";
|
||||
}
|
||||
try {
|
||||
MessageDigest md5 = MessageDigest.getInstance(MD5_ALGORITHM);
|
||||
byte[] encryption = md5.digest(value.getBytes(StandardCharsets.UTF_8));
|
||||
StringBuilder strBuf = new StringBuilder();
|
||||
for (byte b : encryption) {
|
||||
strBuf.append(String.format("%02x", b));
|
||||
}
|
||||
return strBuf.toString();
|
||||
} catch (NoSuchAlgorithmException e) {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* The rules of generating taskId
|
||||
* @param addTaskRequest
|
||||
* @return
|
||||
* Generates a task ID based on the given TaskAddRequest.
|
||||
*
|
||||
* @param addTaskRequest The TaskAddRequest containing task details.
|
||||
* @return The generated task ID.
|
||||
*/
|
||||
public static String generateTaskId(TaskAddRequest addTaskRequest) {
|
||||
|
||||
return generateTaskId(addTaskRequest.getServiceName(), addTaskRequest.getGroupName(),
|
||||
addTaskRequest.getSourceClusterId(), addTaskRequest.getDestClusterId());
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* The rules of generating taskId
|
||||
* Generates a task ID based on the given parameters.
|
||||
*
|
||||
* @return
|
||||
* @param serviceName The service name.
|
||||
* @param groupName The group name.
|
||||
* @param sourceClusterId The source cluster ID.
|
||||
* @param destClusterId The destination cluster ID.
|
||||
* @return The generated task ID.
|
||||
*/
|
||||
public static String generateTaskId(String serviceName, String groupName,
|
||||
String sourceClusterId, String destClusterId) {
|
||||
|
||||
StringBuilder sb = new StringBuilder();
|
||||
|
||||
sb.append(serviceName);
|
||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
||||
sb.append(groupName);
|
||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
||||
sb.append(sourceClusterId);
|
||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
||||
sb.append(destClusterId);
|
||||
return SkyWalkerUtil.StringToMd5(sb.toString());
|
||||
String sourceClusterId, String destClusterId) {
|
||||
String rawId = String.join(SkyWalkerConstants.UNDERLINE, serviceName, groupName, sourceClusterId, destClusterId);
|
||||
return stringToMd5(rawId);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 生成集群clusterId的规则
|
||||
* Generates a cluster ID based on the given ClusterAddRequest.
|
||||
*
|
||||
* @param addClusterRequest
|
||||
* @return
|
||||
* @param addClusterRequest The ClusterAddRequest containing cluster details.
|
||||
* @return The generated cluster ID.
|
||||
*/
|
||||
public static String generateClusterId(ClusterAddRequest addClusterRequest) {
|
||||
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append(addClusterRequest.getClusterName());
|
||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
||||
sb.append(addClusterRequest.getClusterType());
|
||||
|
||||
return SkyWalkerUtil.StringToMd5(sb.toString());
|
||||
String rawId = String.join(SkyWalkerConstants.UNDERLINE, addClusterRequest.getClusterName(), addClusterRequest.getClusterType());
|
||||
return stringToMd5(rawId);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Avoid getting a return address
|
||||
* @return
|
||||
* @throws Exception
|
||||
* Gets the local IP address, avoiding loopback addresses.
|
||||
*
|
||||
* @return The local IP address.
|
||||
* @throws Exception If an error occurs while fetching the IP address.
|
||||
*/
|
||||
public static String getLocalIp() throws Exception {
|
||||
|
||||
InetAddress addr = InetAddress.getLocalHost();
|
||||
String localIp = addr.getHostAddress();
|
||||
if (addr.isLoopbackAddress()) {
|
||||
Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
|
||||
while (interfaces.hasMoreElements()) {
|
||||
NetworkInterface in = interfaces.nextElement();
|
||||
Enumeration<InetAddress> addrs = in.getInetAddresses();
|
||||
while (addrs.hasMoreElements()) {
|
||||
InetAddress address = addrs.nextElement();
|
||||
if (!address.isLoopbackAddress() && address instanceof Inet4Address) {
|
||||
localIp = address.getHostAddress();
|
||||
}
|
||||
if (!addr.isLoopbackAddress()) {
|
||||
return addr.getHostAddress();
|
||||
}
|
||||
Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
|
||||
while (interfaces.hasMoreElements()) {
|
||||
NetworkInterface networkInterface = interfaces.nextElement();
|
||||
Enumeration<InetAddress> addrs = networkInterface.getInetAddresses();
|
||||
while (addrs.hasMoreElements()) {
|
||||
InetAddress address = addrs.nextElement();
|
||||
if (!address.isLoopbackAddress() && address instanceof Inet4Address) {
|
||||
return address.getHostAddress();
|
||||
}
|
||||
}
|
||||
}
|
||||
return localIp;
|
||||
return addr.getHostAddress();
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Generates a synchronization key based on source and destination cluster types.
|
||||
*
|
||||
* @param sourceClusterType The source cluster type.
|
||||
* @param destClusterType The destination cluster type.
|
||||
* @return The generated synchronization key.
|
||||
*/
|
||||
public static String generateSyncKey(ClusterTypeEnum sourceClusterType, ClusterTypeEnum destClusterType) {
|
||||
|
||||
return Joiner.on(":").join(sourceClusterType.getCode(), destClusterType.getCode());
|
||||
return Joiner.on(SEPARATOR).join(sourceClusterType.getCode(), destClusterType.getCode());
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Gets the operation ID from the given TaskDO.
|
||||
*
|
||||
* @param taskDO The TaskDO containing the operation ID.
|
||||
* @return The operation ID.
|
||||
*/
|
||||
public static String getOperationId(TaskDO taskDO) {
|
||||
|
||||
return taskDO.getOperationId();
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Generates a unique operation ID.
|
||||
*
|
||||
* @return The generated operation ID.
|
||||
*/
|
||||
public static String generateOperationId() {
|
||||
|
||||
return UUID.randomUUID().toString();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
|
|
@ -13,19 +13,23 @@
|
|||
package com.alibaba.nacossync.util;
|
||||
|
||||
import com.google.common.base.Joiner;
|
||||
import com.google.common.collect.Maps;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.io.File;
|
||||
import java.io.UnsupportedEncodingException;
|
||||
import java.net.URLDecoder;
|
||||
import java.net.URLEncoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import static com.alibaba.nacossync.util.DubboConstants.*;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_PATH_FORMAT;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_URL_FORMAT;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_IP_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_PORT_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.INTERFACE_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.PROTOCOL_KEY;
|
||||
import static com.alibaba.nacossync.util.DubboConstants.ZOOKEEPER_SEPARATOR;
|
||||
|
||||
/**
|
||||
* @author paderlol
|
||||
|
@ -35,24 +39,26 @@ import static com.alibaba.nacossync.util.DubboConstants.*;
|
|||
public final class StringUtils {
|
||||
|
||||
private static final Pattern KVP_PATTERN = Pattern
|
||||
.compile("([_.a-zA-Z0-9][-_.a-zA-Z0-9]*)[=](.*)");
|
||||
.compile("([_.a-zA-Z0-9][-_.a-zA-Z0-9]*)[=](.*)");
|
||||
private static final Pattern IP_PORT_PATTERN = Pattern
|
||||
.compile(".*/(.*)://(\\d+\\.\\d+\\.\\d+\\.\\d+):(\\d+)");
|
||||
private static final Pattern DUBBO_PROVIDER_PATTERN = Pattern
|
||||
.compile("/dubbo/(.*)/providers/(.*)");
|
||||
.compile(".*/(.*)://(\\d+\\.\\d+\\.\\d+\\.\\d+):(\\d+)");
|
||||
private static final Pattern DUBBO_PROVIDER_PATTERN = Pattern
|
||||
.compile("/dubbo/(.*)/providers/(.*)");
|
||||
public static final int INDEX_NOT_FOUND = -1;
|
||||
public static final String EMPTY = "";
|
||||
|
||||
/**
|
||||
* parse key-value pair.
|
||||
*
|
||||
* @param str string.
|
||||
* @param str string.
|
||||
* @param itemSeparator item separator.
|
||||
* @return key-value map;
|
||||
*/
|
||||
private static Map<String, String> parseKeyValuePair(String str, String itemSeparator) {
|
||||
String[] tmp = str.split(itemSeparator);
|
||||
Map<String, String> map = new HashMap<String, String>(tmp.length);
|
||||
for (int i = 0; i < tmp.length; i++) {
|
||||
Matcher matcher = KVP_PATTERN.matcher(tmp[i]);
|
||||
Map<String, String> map = new HashMap<>(tmp.length);
|
||||
for (String s : tmp) {
|
||||
Matcher matcher = KVP_PATTERN.matcher(s);
|
||||
if (!matcher.matches()) {
|
||||
continue;
|
||||
}
|
||||
|
@ -68,17 +74,14 @@ public final class StringUtils {
|
|||
* @return Parameters instance.
|
||||
*/
|
||||
public static Map<String, String> parseQueryString(String qs) {
|
||||
try {
|
||||
String decodePath = URLDecoder.decode(qs, "UTF-8");
|
||||
if (isEmpty(qs)) {
|
||||
return new HashMap<>();
|
||||
}
|
||||
return parseKeyValuePair(decodePath, "\\&");
|
||||
|
||||
} catch (UnsupportedEncodingException e) {
|
||||
log.warn("parse query string failed", e);
|
||||
return Maps.newHashMap();
|
||||
|
||||
String decodePath = URLDecoder.decode(qs, StandardCharsets.UTF_8);
|
||||
if (isEmpty(decodePath)) {
|
||||
return new HashMap<>();
|
||||
}
|
||||
decodePath = substringAfter(decodePath, "?");
|
||||
return parseKeyValuePair(decodePath, "&");
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
|
@ -92,49 +95,79 @@ public final class StringUtils {
|
|||
}
|
||||
|
||||
public static Map<String, String> parseIpAndPortString(String path) {
|
||||
|
||||
try {
|
||||
String decodePath = URLDecoder.decode(path, "UTF-8");
|
||||
Matcher matcher = IP_PORT_PATTERN.matcher(decodePath);
|
||||
// extract the ones that match the rules
|
||||
Map<String, String> instanceMap = new HashMap<>(3);
|
||||
while (matcher.find()) {
|
||||
// protocol
|
||||
instanceMap.put(PROTOCOL_KEY, matcher.group(1));
|
||||
// ip address
|
||||
instanceMap.put(INSTANCE_IP_KEY, matcher.group(2));
|
||||
// port
|
||||
instanceMap.put(INSTANCE_PORT_KEY, matcher.group(3));
|
||||
break;
|
||||
}
|
||||
return instanceMap;
|
||||
} catch (UnsupportedEncodingException e) {
|
||||
log.warn("parse query string failed", e);
|
||||
return Maps.newHashMap();
|
||||
|
||||
String decodePath = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
||||
Matcher matcher = IP_PORT_PATTERN.matcher(decodePath);
|
||||
// extract the ones that match the rules
|
||||
Map<String, String> instanceMap = new HashMap<>(3);
|
||||
while (matcher.find()) {
|
||||
// protocol
|
||||
instanceMap.put(PROTOCOL_KEY, matcher.group(1));
|
||||
// ip address
|
||||
instanceMap.put(INSTANCE_IP_KEY, matcher.group(2));
|
||||
// port
|
||||
instanceMap.put(INSTANCE_PORT_KEY, matcher.group(3));
|
||||
break;
|
||||
}
|
||||
return instanceMap;
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* <p>Gets the substring after the first occurrence of a separator.
|
||||
* The separator is not returned.</p>
|
||||
*
|
||||
* <p>A {@code null} string input will return {@code null}.
|
||||
* An empty ("") string input will return the empty string. A {@code null} separator will return the empty string if
|
||||
* the input string is not {@code null}.</p>
|
||||
*
|
||||
* <p>If nothing is found, the empty string is returned.</p>
|
||||
*
|
||||
* <pre>
|
||||
* StringUtils.substringAfter(null, *) = null
|
||||
* StringUtils.substringAfter("", *) = ""
|
||||
* StringUtils.substringAfter(*, null) = ""
|
||||
* StringUtils.substringAfter("abc", "a") = "bc"
|
||||
* StringUtils.substringAfter("abcba", "b") = "cba"
|
||||
* StringUtils.substringAfter("abc", "c") = ""
|
||||
* StringUtils.substringAfter("abc", "d") = ""
|
||||
* StringUtils.substringAfter("abc", "") = "abc"
|
||||
* </pre>
|
||||
*
|
||||
* @param str the String to get a substring from, may be null
|
||||
* @param separator the String to search for, may be null
|
||||
* @return the substring after the first occurrence of the separator, {@code null} if null String input
|
||||
* @since 2.0
|
||||
*/
|
||||
public static String substringAfter(final String str, final String separator) {
|
||||
if (isEmpty(str)) {
|
||||
return str;
|
||||
}
|
||||
if (separator == null) {
|
||||
return EMPTY;
|
||||
}
|
||||
final int pos = str.indexOf(separator);
|
||||
if (pos == INDEX_NOT_FOUND) {
|
||||
return EMPTY;
|
||||
}
|
||||
return str.substring(pos + 1);
|
||||
}
|
||||
|
||||
public static String convertDubboProvidersPath(String interfaceName) {
|
||||
return String.format(DUBBO_PATH_FORMAT, interfaceName);
|
||||
}
|
||||
|
||||
public static String convertDubboFullPathForZk(Map<String, String> metaData, String providersPath, String ip,
|
||||
int port) {
|
||||
try {
|
||||
String urlParam = Joiner.on("&").withKeyValueSeparator("=").join(metaData);
|
||||
String instanceUrl = String.format(DUBBO_URL_FORMAT, metaData.get(PROTOCOL_KEY), ip, port,
|
||||
metaData.get(INTERFACE_KEY), urlParam);
|
||||
public static String convertDubboFullPathForZk(Map<String, String> metaData, String providersPath, String ip,
|
||||
int port) {
|
||||
String urlParam = Joiner.on("&").withKeyValueSeparator("=").join(metaData);
|
||||
String instanceUrl = String.format(DUBBO_URL_FORMAT, metaData.get(PROTOCOL_KEY), ip, port,
|
||||
metaData.get(INTERFACE_KEY), urlParam);
|
||||
|
||||
return Joiner.on(ZOOKEEPER_SEPARATOR).join(providersPath, URLEncoder.encode(instanceUrl, StandardCharsets.UTF_8));
|
||||
|
||||
}
|
||||
|
||||
return Joiner.on(ZOOKEEPER_SEPARATOR).join(providersPath, URLEncoder.encode(instanceUrl, "UTF-8"));
|
||||
} catch (UnsupportedEncodingException e) {
|
||||
log.warn("convert Dubbo full path", e);
|
||||
return "";
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
public static boolean isDubboProviderPath(String path) {
|
||||
return DUBBO_PROVIDER_PATTERN.matcher(path).matches();
|
||||
}
|
||||
public static boolean isDubboProviderPath(String path) {
|
||||
return DUBBO_PROVIDER_PATTERN.matcher(path).matches();
|
||||
}
|
||||
}
|
||||
|
|
|
@ -0,0 +1,207 @@
|
|||
/*
|
||||
* Copyright (c) 2016-2021 VMware Inc. or its affiliates, All Rights Reserved.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.alibaba.nacossync.util;
|
||||
|
||||
import reactor.util.annotation.NonNull;
|
||||
import reactor.util.annotation.Nullable;
|
||||
import reactor.util.function.Tuple2;
|
||||
import reactor.util.function.Tuples;
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.Arrays;
|
||||
import java.util.Collections;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.function.Function;
|
||||
|
||||
/**
|
||||
* A tuple that holds two non-null values.
|
||||
*
|
||||
* @param <T1> The type of the first non-null value held by this tuple
|
||||
* @param <T2> The type of the second non-null value held by this tuple
|
||||
* @author Jon Brisbin
|
||||
* @author Stephane Maldini
|
||||
*/
|
||||
@SuppressWarnings("rawtypes")
|
||||
public class Tuple<T1, T2> implements Iterable<Object>, Serializable {
|
||||
|
||||
private static final long serialVersionUID = -3518082018884860684L;
|
||||
|
||||
@NonNull
|
||||
final T1 t1;
|
||||
|
||||
@NonNull
|
||||
final T2 t2;
|
||||
|
||||
Tuple(T1 t1, T2 t2) {
|
||||
this.t1 = Objects.requireNonNull(t1, "t1");
|
||||
this.t2 = Objects.requireNonNull(t2, "t2");
|
||||
}
|
||||
|
||||
/**
|
||||
* Type-safe way to get the first object of this {@link Tuples}.
|
||||
*
|
||||
* @return The first object
|
||||
*/
|
||||
public T1 getT1() {
|
||||
return t1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Type-safe way to get the second object of this {@link Tuples}.
|
||||
*
|
||||
* @return The second object
|
||||
*/
|
||||
public T2 getT2() {
|
||||
return t2;
|
||||
}
|
||||
|
||||
/**
|
||||
* Map the left-hand part (T1) of this {@link Tuple} into a different value and type, keeping the right-hand part
|
||||
* (T2).
|
||||
*
|
||||
* @param mapper the mapping {@link Function} for the left-hand part
|
||||
* @param <R> the new type for the left-hand part
|
||||
* @return a new {@link Tuple2} with a different left (T1) value
|
||||
*/
|
||||
public <R> Tuple<R, T2> mapT1(Function<T1, R> mapper) {
|
||||
return new Tuple<>(mapper.apply(t1), t2);
|
||||
}
|
||||
|
||||
/**
|
||||
* Map the right-hand part (T2) of this {@link Tuple} into a different value and type, keeping the left-hand part
|
||||
* (T1).
|
||||
*
|
||||
* @param mapper the mapping {@link Function} for the right-hand part
|
||||
* @param <R> the new type for the right-hand part
|
||||
* @return a new {@link Tuple2} with a different right (T2) value
|
||||
*/
|
||||
public <R> Tuple<T1, R> mapT2(Function<T2, R> mapper) {
|
||||
return new Tuple<>(t1, mapper.apply(t2));
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the object at the given index.
|
||||
*
|
||||
* @param index The index of the object to retrieve. Starts at 0.
|
||||
* @return The object or {@literal null} if out of bounds.
|
||||
*/
|
||||
@Nullable
|
||||
public Object get(int index) {
|
||||
switch (index) {
|
||||
case 0:
|
||||
return t1;
|
||||
case 1:
|
||||
return t2;
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Turn this {@code Tuple} into a {@link List List<Object>}. The list isn't tied to this Tuple but is a
|
||||
* <strong>copy</strong> with limited mutability ({@code add} and {@code remove} are not supported, but {@code set}
|
||||
* is).
|
||||
*
|
||||
* @return A copy of the tuple as a new {@link List List<Object>}.
|
||||
*/
|
||||
public List<Object> toList() {
|
||||
return Arrays.asList(toArray());
|
||||
}
|
||||
|
||||
/**
|
||||
* Turn this {@code Tuple} into a plain {@code Object[]}. The array isn't tied to this Tuple but is a
|
||||
* <strong>copy</strong>.
|
||||
*
|
||||
* @return A copy of the tuple as a new {@link Object Object[]}.
|
||||
*/
|
||||
public Object[] toArray() {
|
||||
return new Object[] {t1, t2};
|
||||
}
|
||||
|
||||
/**
|
||||
* Return an <strong>immutable</strong> {@link Iterator Iterator<Object>} around the content of this
|
||||
* {@code Tuple}.
|
||||
*
|
||||
* @return An unmodifiable {@link Iterator} over the elements in this Tuple.
|
||||
* @implNote As an {@link Iterator} is always tied to its {@link Iterable} source by definition, the iterator cannot
|
||||
* be mutable without the iterable also being mutable. Since {@link Tuples} are <strong>immutable</strong>, so is
|
||||
* the {@link Iterator} returned by this method.
|
||||
*/
|
||||
@Override
|
||||
public Iterator<Object> iterator() {
|
||||
return Collections.unmodifiableList(toList()).iterator();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean equals(@Nullable Object o) {
|
||||
if (this == o) {
|
||||
return true;
|
||||
}
|
||||
if (o == null || getClass() != o.getClass()) {
|
||||
return false;
|
||||
}
|
||||
|
||||
Tuple<?, ?> tuple = (Tuple<?, ?>) o;
|
||||
|
||||
return t1.equals(tuple.t1) && t2.equals(tuple.t2);
|
||||
|
||||
}
|
||||
|
||||
@Override
|
||||
public int hashCode() {
|
||||
int result = size();
|
||||
result = 31 * result + t1.hashCode();
|
||||
result = 31 * result + t2.hashCode();
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the number of elements in this {@literal Tuples}.
|
||||
*
|
||||
* @return The size of this {@literal Tuples}.
|
||||
*/
|
||||
public int size() {
|
||||
return 2;
|
||||
}
|
||||
|
||||
/**
|
||||
* A Tuple String representation is the comma separated list of values, enclosed in square brackets.
|
||||
*
|
||||
* @return the Tuple String representation
|
||||
*/
|
||||
@Override
|
||||
public final String toString() {
|
||||
Object[] values = toArray();
|
||||
StringBuilder sb = new StringBuilder();
|
||||
for (int i = 0; i < values.length; i++) {
|
||||
Object t = values[i];
|
||||
if (i != 0) {
|
||||
sb.append(',');
|
||||
}
|
||||
if (t != null) {
|
||||
sb.append(t);
|
||||
}
|
||||
}
|
||||
return sb.insert(0, '[').append(']').toString();
|
||||
}
|
||||
|
||||
public static <T1, T2> Tuple<T1, T2> of(T1 t1, T2 t2) {
|
||||
return new Tuple<>(t1, t2);
|
||||
}
|
||||
}
|
|
@ -5,11 +5,11 @@ spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
|
|||
spring.jpa.hibernate.ddl-auto=update
|
||||
spring.jpa.properties.hibernate.show_sql=false
|
||||
|
||||
spring.cloud.discovery.enabled=false
|
||||
|
||||
|
||||
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/nacos_sync?characterEncoding=utf8
|
||||
spring.datasource.username=root
|
||||
spring.datasource.password=root
|
||||
|
||||
management.endpoints.web.exposure.include=*
|
||||
management.endpoint.health.show-details=always
|
||||
|
|
|
@ -4,12 +4,13 @@
|
|||
<springProperty name="logPath" scope="context" source="nacosSync.logs.path" defaultValue="${nacosSync.home}/logs"/>
|
||||
<property name="LOG_HOME" value="${logPath}" />
|
||||
<appender name="FILEINFO" class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||
<file>${LOG_HOME}/nacosSync.log</file>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
|
||||
<fileNamePattern>${LOG_HOME}/log-nacosSync-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
|
||||
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
|
||||
<maxFileSize>200MB</maxFileSize>
|
||||
</timeBasedFileNamingAndTriggeringPolicy>
|
||||
<file>${LOG_HOME}/nacos-sync.log</file>
|
||||
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
|
||||
<fileNamePattern>${LOG_HOME}/log-nacos-sync-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
|
||||
<maxFileSize>200MB</maxFileSize>
|
||||
<maxHistory>7</maxHistory>
|
||||
<totalSizeCap>2GB</totalSizeCap>
|
||||
<cleanHistoryOnStart>true</cleanHistoryOnStart>
|
||||
</rollingPolicy>
|
||||
<append>true</append>
|
||||
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
|
||||
|
@ -25,7 +26,6 @@
|
|||
</appender>
|
||||
|
||||
<logger name="com.alibaba.nacossync" level="INFO"/>
|
||||
|
||||
<logger name="org.hibernate.type.descriptor.sql.BasicBinder" level="INFO"/>
|
||||
<logger name="org.hibernate.type.descriptor.sql.BasicExtractor" level="INFO"/>
|
||||
<logger name="org.hibernate.SQL" level="INFO"/>
|
||||
|
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
@ -7,8 +7,8 @@
|
|||
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
||||
<title>Nacos-Sync</title>
|
||||
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
||||
<link href="./css/main.68bce23a.css" rel="stylesheet"></head>
|
||||
<link href="./css/main.e2917886.css" rel="stylesheet"></head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="text/javascript" src="./js/main.1cd0c600.js"></script></body>
|
||||
<script type="text/javascript" src="./js/main.b73436b5.js"></script></body>
|
||||
</html>
|
||||
|
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue