Compare commits
95 Commits
Author | SHA1 | Date |
---|---|---|
|
0c35e096c2 | |
|
142a36f608 | |
|
b18afb95c1 | |
|
aa87b24e1a | |
|
fdc51ba605 | |
|
ce6249142f | |
|
26c8b307d7 | |
|
6585b217bc | |
|
b609c729c3 | |
|
432ced4e0e | |
|
116116c8c4 | |
|
ca08bd7d6e | |
|
81610e505a | |
|
17439be724 | |
|
91999fbdcb | |
|
6645b00f24 | |
|
acb7b49551 | |
|
ba416d1496 | |
|
ff5e484997 | |
|
b900c28347 | |
|
5d17242ffc | |
|
39357660ab | |
|
49abb7c286 | |
|
8185feca1d | |
|
5789d4d774 | |
|
2fdf44d308 | |
|
5941bfb9bc | |
|
0c24dcb0a2 | |
|
a1d683a7c9 | |
|
be65db1892 | |
|
eb93fdd52a | |
|
e7e52acfa7 | |
|
02838da959 | |
|
0e73a0864c | |
|
61d347609e | |
|
a9df169b5a | |
|
98efe26906 | |
|
384a9cc1bb | |
|
0ffdca3e0b | |
|
5ae4071508 | |
|
4082efa1fc | |
|
fc03e6483c | |
|
2b8d29a0ab | |
|
881c11d51f | |
|
855922964e | |
|
9e814d2d43 | |
|
cdfbebf407 | |
|
017cacf12c | |
|
ab4a4bbb92 | |
|
853119b615 | |
|
f99398c649 | |
|
a61a79aca9 | |
|
00570523fa | |
|
508a912c35 | |
|
3ee0d33fff | |
|
03e4419157 | |
|
3998dfedf1 | |
|
1ffec6a785 | |
|
0fad6ef8cc | |
|
46b3590fb6 | |
|
a2d870e15e | |
|
758b18cb7d | |
|
d630fb16b4 | |
|
7c6f889e72 | |
|
0826dc9c15 | |
|
96111de3da | |
|
137fdee7f1 | |
|
c4200d7516 | |
|
30b331b4b0 | |
|
8fceffde07 | |
|
8dde539a79 | |
|
1cd2e3b408 | |
|
ee3040f011 | |
|
c5d574e357 | |
|
eb3e74a678 | |
|
1f16efb043 | |
|
467752e355 | |
|
91bf0d93f8 | |
|
9cab680e46 | |
|
c62e93a677 | |
|
e3354fefb0 | |
|
57e03c8aee | |
|
f1c1c00190 | |
|
4fced18886 | |
|
bdfa05670e | |
|
d389587d96 | |
|
e5283d2a23 | |
|
e65fba55a0 | |
|
b17f6fff8d | |
|
8748f0c2be | |
|
c38eed392d | |
|
070ac65f78 | |
|
a48db6ec63 | |
|
a6f8708aa0 | |
|
98fad6eb9d |
|
@ -11,3 +11,4 @@ target
|
||||||
node_modules
|
node_modules
|
||||||
test/derby.log
|
test/derby.log
|
||||||
/console-fe
|
/console-fe
|
||||||
|
.flattened-pom.xml
|
||||||
|
|
41
README.md
41
README.md
|
@ -1,9 +1,11 @@
|
||||||
# Nacos Sync
|
# Nacos Sync
|
||||||
|
|
||||||
|
## [Example](https://github.com/paderlol/nacos-sync-example)
|
||||||
|
|
||||||
## Function
|
## Function
|
||||||
|
|
||||||
- Console: provide API and console for management
|
- Console: provide API and console for management
|
||||||
- Worker: provider the service registration synchronization.
|
- Worker: provide the service registration synchronization.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
|
@ -42,15 +44,9 @@ Info | +------------+ ^
|
||||||
- NacosCluster target will dedup the synchronization information from Nacos.
|
- NacosCluster target will dedup the synchronization information from Nacos.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## Quick Start:
|
## Quick Start:
|
||||||
- Swagger API: http://127.0.0.1:8081/swagger-ui.html#/
|
- Swagger API: http://127.0.0.1:8083/swagger-ui.html#/
|
||||||
- Web Console: http://127.0.0.1:8081/
|
- Web Console: http://127.0.0.1:8083/
|
||||||
- Others: TBD
|
- Others: TBD
|
||||||
|
|
||||||
# NacosSync Migration User Guide
|
# NacosSync Migration User Guide
|
||||||
|
@ -82,7 +78,7 @@ Before you begin, install the following:
|
||||||
|
|
||||||
- 64bit OS: Linux/Unix/Mac/Windows supported, Linux/Unix/Mac recommended.
|
- 64bit OS: Linux/Unix/Mac/Windows supported, Linux/Unix/Mac recommended.
|
||||||
- 64bit JDK 1.8+: downloads, JAVA_HOME settings.
|
- 64bit JDK 1.8+: downloads, JAVA_HOME settings.
|
||||||
- Maven 3.2.x+: downloads, settings.
|
- Maven 3.5.2+: [downloads](https://maven.apache.org/download.cgi), [settings](https://maven.apache.org/settings.html).
|
||||||
- MySql 5.6.+
|
- MySql 5.6.+
|
||||||
|
|
||||||
## Download & Build From Release
|
## Download & Build From Release
|
||||||
|
@ -94,7 +90,7 @@ There are two ways to get NacosSync.
|
||||||
|
|
||||||
``` xml
|
``` xml
|
||||||
|
|
||||||
cd nacosSync/
|
cd nacos-sync/
|
||||||
mvn clean package -U
|
mvn clean package -U
|
||||||
|
|
||||||
```
|
```
|
||||||
|
@ -103,7 +99,7 @@ The path to the target file:
|
||||||
|
|
||||||
``` xml
|
``` xml
|
||||||
|
|
||||||
nacos-sync/nacossync-distribution/target/nacosSync.0.3.8.zip
|
nacos-sync/nacossync-distribution/target/nacos-sync-0.5.0.tar.gz
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -111,7 +107,7 @@ After extracting the installation package, the directory structure:
|
||||||
|
|
||||||
``` xml
|
``` xml
|
||||||
|
|
||||||
nacosSync
|
nacos-sync
|
||||||
├── LICENSE
|
├── LICENSE
|
||||||
├── NOTICE
|
├── NOTICE
|
||||||
├── bin
|
├── bin
|
||||||
|
@ -122,7 +118,7 @@ nacosSync
|
||||||
│ ├── application.properties
|
│ ├── application.properties
|
||||||
│ └── logback-spring.xml
|
│ └── logback-spring.xml
|
||||||
├── logs
|
├── logs
|
||||||
└── nacosSync-server.jar
|
└── nacos-sync-server.jar
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -130,7 +126,7 @@ nacosSync
|
||||||
|
|
||||||
The default is Mysql database, which can support other relational databases
|
The default is Mysql database, which can support other relational databases
|
||||||
|
|
||||||
- Build db schema, the default schema name nacos_Sync.
|
- Build db schema, the default schema name nacos_sync.
|
||||||
- Tables do not need to be created separately, which is conducive to hibernate's automatic table creation function.
|
- Tables do not need to be created separately, which is conducive to hibernate's automatic table creation function.
|
||||||
- If the automatic table creation fails, you can build the table nacosSync.sql, the table statement is in the bin folder.
|
- If the automatic table creation fails, you can build the table nacosSync.sql, the table statement is in the bin folder.
|
||||||
|
|
||||||
|
@ -159,15 +155,18 @@ sh startup.sh start
|
||||||
|
|
||||||
``` xml
|
``` xml
|
||||||
|
|
||||||
http://127.0.0.1:8081/#/serviceSync
|
http://127.0.0.1:8083/#/serviceSync
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Advanced Configuration
|
||||||
|
|
||||||
|
### Full Synchronization from Zookeeper to Nacos (Dubbo)
|
||||||
|
When “*” is entered in the “Service Name” field of this form, it will fully synchronize all services from Zookeeper to Nacos, but only when using Dubbo.
|
||||||
|

|
||||||
|
|
||||||
|
### Full Synchronization from Nacos to Nacos
|
||||||
|
When “All” is entered in the “Service Name” field of this form, it will automatically synchronize all registered services within the **default group** of the current cluster.
|
||||||
|
|
||||||
|
This description explains the functionality clearly for English-speaking users.
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -16,7 +16,7 @@
|
||||||
<parent>
|
<parent>
|
||||||
<artifactId>nacossync-parent</artifactId>
|
<artifactId>nacossync-parent</artifactId>
|
||||||
<groupId>com.alibaba.nacossync</groupId>
|
<groupId>com.alibaba.nacossync</groupId>
|
||||||
<version>0.4.4</version>
|
<version>${revision}</version>
|
||||||
</parent>
|
</parent>
|
||||||
<modelVersion>4.0.0</modelVersion>
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
|
||||||
|
@ -27,5 +27,4 @@
|
||||||
<artifactId>junit</artifactId>
|
<artifactId>junit</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
</dependencies>
|
</dependencies>
|
||||||
|
|
||||||
</project>
|
</project>
|
File diff suppressed because one or more lines are too long
|
@ -7,8 +7,8 @@
|
||||||
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
||||||
<title>Nacos-Sync</title>
|
<title>Nacos-Sync</title>
|
||||||
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
||||||
<link href="./css/main.68bce23a.css" rel="stylesheet"></head>
|
<link href="./css/main.e2917886.css" rel="stylesheet"></head>
|
||||||
<body>
|
<body>
|
||||||
<div id="root"></div>
|
<div id="root"></div>
|
||||||
<script type="text/javascript" src="./js/main.1cd0c600.js"></script></body>
|
<script type="text/javascript" src="./js/main.b73436b5.js"></script></body>
|
||||||
</html>
|
</html>
|
||||||
|
|
File diff suppressed because one or more lines are too long
|
@ -18,6 +18,9 @@ class AddConfigDialog extends React.Component {
|
||||||
visible: false,
|
visible: false,
|
||||||
clusterName: '',
|
clusterName: '',
|
||||||
clusterType: '',
|
clusterType: '',
|
||||||
|
namespace: '',
|
||||||
|
password: '',
|
||||||
|
userName: '',
|
||||||
connectKeyList: [],
|
connectKeyList: [],
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
@ -27,8 +30,8 @@ class AddConfigDialog extends React.Component {
|
||||||
}
|
}
|
||||||
|
|
||||||
save() {
|
save() {
|
||||||
const { clusterName, clusterType, connectKeyList } = this.state;
|
const { clusterName, namespace, userName, password, clusterType, connectKeyList } = this.state;
|
||||||
add({ clusterName, clusterType, connectKeyList })
|
add({ clusterName, namespace, userName, password, clusterType, connectKeyList })
|
||||||
.then(() => {
|
.then(() => {
|
||||||
this.props.turnPage(1);
|
this.props.turnPage(1);
|
||||||
this.close();
|
this.close();
|
||||||
|
@ -37,7 +40,10 @@ class AddConfigDialog extends React.Component {
|
||||||
}
|
}
|
||||||
|
|
||||||
close() {
|
close() {
|
||||||
this.setState({ visible: false });
|
this.setState({
|
||||||
|
visible: false,
|
||||||
|
clusterType: '',
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
open = () => this.setState({ visible: true })
|
open = () => this.setState({ visible: true })
|
||||||
|
@ -71,6 +77,35 @@ class AddConfigDialog extends React.Component {
|
||||||
}
|
}
|
||||||
</Select>
|
</Select>
|
||||||
</FormItem>
|
</FormItem>
|
||||||
|
{
|
||||||
|
this.state.clusterType === 'NACOS' && (
|
||||||
|
<>
|
||||||
|
<FormItem
|
||||||
|
label={`${locale.namespace}:`}
|
||||||
|
>
|
||||||
|
<Input
|
||||||
|
placeholder={locale.namespacePlaceholder}
|
||||||
|
onChange={ns => this.setState({ namespace: ns })}
|
||||||
|
/>
|
||||||
|
</FormItem>
|
||||||
|
<FormItem
|
||||||
|
label={`${locale.username}:`}
|
||||||
|
>
|
||||||
|
<Input
|
||||||
|
placeholder={locale.usernamePlaceholder}
|
||||||
|
onChange={un => this.setState({ userName: un })}
|
||||||
|
/>
|
||||||
|
</FormItem>
|
||||||
|
<FormItem
|
||||||
|
label={`${locale.password}:`}
|
||||||
|
>
|
||||||
|
<Input.Password
|
||||||
|
placeholder={locale.passwordPlaceholder}
|
||||||
|
onChange={pw => this.setState({ password: pw })}
|
||||||
|
/>
|
||||||
|
</FormItem>
|
||||||
|
</>)
|
||||||
|
}
|
||||||
<FormItem label={`${locale.connectKeyList}:`}>
|
<FormItem label={`${locale.connectKeyList}:`}>
|
||||||
<Input.TextArea
|
<Input.TextArea
|
||||||
onChange={(connectKeyListStr) => {
|
onChange={(connectKeyListStr) => {
|
||||||
|
|
|
@ -98,6 +98,7 @@ class ClusterConfig extends React.Component {
|
||||||
<Table.Column title={locale.clusterName} dataIndex="clusterName" />
|
<Table.Column title={locale.clusterName} dataIndex="clusterName" />
|
||||||
<Table.Column title={locale.clusterType} dataIndex="clusterType" />
|
<Table.Column title={locale.clusterType} dataIndex="clusterType" />
|
||||||
<Table.Column title={locale.connectKeyList} dataIndex="connectKeyList" />
|
<Table.Column title={locale.connectKeyList} dataIndex="connectKeyList" />
|
||||||
|
<Table.Column title={locale.namespace} dataIndex="namespace" />
|
||||||
<Table.Column
|
<Table.Column
|
||||||
title={locale.operation}
|
title={locale.operation}
|
||||||
cell={(value, index, record) => (
|
cell={(value, index, record) => (
|
||||||
|
|
|
@ -13,7 +13,7 @@ class Layout extends React.Component {
|
||||||
<Header />
|
<Header />
|
||||||
<Row className="layout">
|
<Row className="layout">
|
||||||
<Col fixedSpan="9" className="nav-bar">
|
<Col fixedSpan="9" className="nav-bar">
|
||||||
<h1 className="title">Nacos-Sync 3.0</h1>
|
<h1 className="title">Nacos-Sync 0.4.8</h1>
|
||||||
<Menu />
|
<Menu />
|
||||||
</Col>
|
</Col>
|
||||||
<Col className="main-panel">{this.props.children}</Col>
|
<Col className="main-panel">{this.props.children}</Col>
|
||||||
|
|
|
@ -19,6 +19,9 @@ const I18N_CONF = {
|
||||||
clusterNamePlaceholder: 'Please enter the cluster name',
|
clusterNamePlaceholder: 'Please enter the cluster name',
|
||||||
clusterType: 'Cluster Type',
|
clusterType: 'Cluster Type',
|
||||||
connectKeyList: 'Connect Key List',
|
connectKeyList: 'Connect Key List',
|
||||||
|
namespace: 'Namespace',
|
||||||
|
password: 'Password',
|
||||||
|
username: 'Username',
|
||||||
operation: 'Operation',
|
operation: 'Operation',
|
||||||
deleteBtn: 'Delete',
|
deleteBtn: 'Delete',
|
||||||
confirm: 'Prompt',
|
confirm: 'Prompt',
|
||||||
|
@ -29,6 +32,8 @@ const I18N_CONF = {
|
||||||
title: 'New Cluster',
|
title: 'New Cluster',
|
||||||
clusterName: 'Cluster Name',
|
clusterName: 'Cluster Name',
|
||||||
clusterNamePlaceholder: 'Please enter the cluster name',
|
clusterNamePlaceholder: 'Please enter the cluster name',
|
||||||
|
namespace: 'Namespace',
|
||||||
|
namespacePlaceholder: 'Please enter the namespace',
|
||||||
clusterType: 'Cluster Type',
|
clusterType: 'Cluster Type',
|
||||||
connectKeyList: 'Connect IP',
|
connectKeyList: 'Connect IP',
|
||||||
connectKeyListPlaceholder: 'Please enter the ip',
|
connectKeyListPlaceholder: 'Please enter the ip',
|
||||||
|
|
|
@ -19,6 +19,7 @@ const I18N_CONF = {
|
||||||
clusterNamePlaceholder: '请输入集群名',
|
clusterNamePlaceholder: '请输入集群名',
|
||||||
clusterType: '集群类型',
|
clusterType: '集群类型',
|
||||||
connectKeyList: '集群IP列表',
|
connectKeyList: '集群IP列表',
|
||||||
|
namespace: '命名空间',
|
||||||
operation: '操作',
|
operation: '操作',
|
||||||
deleteBtn: '删除',
|
deleteBtn: '删除',
|
||||||
confirm: '提示',
|
confirm: '提示',
|
||||||
|
@ -29,6 +30,12 @@ const I18N_CONF = {
|
||||||
title: '新增集群',
|
title: '新增集群',
|
||||||
clusterName: '集群名',
|
clusterName: '集群名',
|
||||||
clusterNamePlaceholder: '请输入集群名',
|
clusterNamePlaceholder: '请输入集群名',
|
||||||
|
namespace: '命名空间',
|
||||||
|
namespacePlaceholder: '请输入命名空间',
|
||||||
|
password: '密码',
|
||||||
|
passwordPlaceholder: '请输入密码',
|
||||||
|
username: '用户名',
|
||||||
|
usernamePlaceholder: '请输入用户名',
|
||||||
clusterType: '集群类型',
|
clusterType: '集群类型',
|
||||||
connectKeyList: '集群IP列表',
|
connectKeyList: '集群IP列表',
|
||||||
connectKeyListPlaceholder: '请输入集群IP',
|
connectKeyListPlaceholder: '请输入集群IP',
|
||||||
|
|
File diff suppressed because one or more lines are too long
|
@ -7,8 +7,8 @@
|
||||||
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
||||||
<title>Nacos-Sync</title>
|
<title>Nacos-Sync</title>
|
||||||
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
||||||
<link href="./css/main.68bce23a.css" rel="stylesheet"></head>
|
<link href="./css/main.e2917886.css" rel="stylesheet"></head>
|
||||||
<body>
|
<body>
|
||||||
<div id="root"></div>
|
<div id="root"></div>
|
||||||
<script type="text/javascript" src="./js/main.1cd0c600.js"></script></body>
|
<script type="text/javascript" src="./js/main.b73436b5.js"></script></body>
|
||||||
</html>
|
</html>
|
||||||
|
|
File diff suppressed because one or more lines are too long
|
@ -1,5 +1,5 @@
|
||||||
/******************************************/
|
/******************************************/
|
||||||
/* DB name = nacos_Sync */
|
/* DB name = nacos_sync */
|
||||||
/* Table name = cluster */
|
/* Table name = cluster */
|
||||||
/******************************************/
|
/******************************************/
|
||||||
CREATE TABLE `cluster` (
|
CREATE TABLE `cluster` (
|
||||||
|
@ -8,10 +8,14 @@ CREATE TABLE `cluster` (
|
||||||
`cluster_name` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`cluster_name` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
`cluster_type` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`cluster_type` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
`connect_key_list` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`connect_key_list` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
|
`user_name` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
|
`password` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
|
`namespace` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
|
`cluster_level` int default 0,
|
||||||
PRIMARY KEY (`id`)
|
PRIMARY KEY (`id`)
|
||||||
) ENGINE=MyISAM AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||||
/******************************************/
|
/******************************************/
|
||||||
/* DB name = nacos_Sync */
|
/* DB name = nacos_sync */
|
||||||
/* Table name = system_config */
|
/* Table name = system_config */
|
||||||
/******************************************/
|
/******************************************/
|
||||||
CREATE TABLE `system_config` (
|
CREATE TABLE `system_config` (
|
||||||
|
@ -20,9 +24,9 @@ CREATE TABLE `system_config` (
|
||||||
`config_key` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`config_key` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
`config_value` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`config_value` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
PRIMARY KEY (`id`)
|
PRIMARY KEY (`id`)
|
||||||
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||||
/******************************************/
|
/******************************************/
|
||||||
/* DB name = nacos_Sync */
|
/* DB name = nacos_sync */
|
||||||
/* Table name = task */
|
/* Table name = task */
|
||||||
/******************************************/
|
/******************************************/
|
||||||
CREATE TABLE `task` (
|
CREATE TABLE `task` (
|
||||||
|
@ -37,5 +41,6 @@ CREATE TABLE `task` (
|
||||||
`task_status` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`task_status` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
`version` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`version` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
`worker_ip` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
`worker_ip` varchar(255) COLLATE utf8mb4_bin DEFAULT NULL,
|
||||||
|
`status` int default null ,
|
||||||
PRIMARY KEY (`id`)
|
PRIMARY KEY (`id`)
|
||||||
) ENGINE=MyISAM AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||||
|
|
|
@ -13,14 +13,14 @@
|
||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
pid=`ps ax | grep -i 'nacosSync' |grep java | grep -v grep | awk '{print $1}'`
|
pid=`ps ax | grep -i 'nacos-sync' |grep java | grep -v grep | awk '{print $1}'`
|
||||||
if [ -z "$pid" ] ; then
|
if [ -z "$pid" ] ; then
|
||||||
echo "No nacosSync running."
|
echo "no nacos-sync running."
|
||||||
exit -1;
|
exit -1;
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "The nacosSync(${pid}) is running..."
|
echo "the nacos-sync(${pid}) is running..."
|
||||||
|
|
||||||
kill ${pid}
|
kill ${pid}
|
||||||
|
|
||||||
echo "Send shutdown request to nacosSync(${pid}) OK"
|
echo "Send shutdown request to nacos-sync(${pid}) OK"
|
|
@ -0,0 +1 @@
|
||||||
|
java -jar ../nacos-sync-server.jar -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m --spring.config.location=../conf/application.properties
|
|
@ -47,24 +47,24 @@ export BASE_DIR=`cd $(dirname $0)/..; pwd`
|
||||||
|
|
||||||
|
|
||||||
JAVA_OPT="${JAVA_OPT} -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m"
|
JAVA_OPT="${JAVA_OPT} -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m"
|
||||||
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${BASE_DIR}/nacossync_java_heapdump.hprof"
|
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${BASE_DIR}/nacos-sync-java-heapdump.hprof"
|
||||||
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"
|
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"
|
||||||
JAVA_OPT="${JAVA_OPT} -Dspring.config.location=${BASE_DIR}/conf/application.properties"
|
JAVA_OPT="${JAVA_OPT} -Dspring.config.location=${BASE_DIR}/conf/application.properties"
|
||||||
JAVA_OPT="${JAVA_OPT} -DnacosSync.home=${BASE_DIR}"
|
JAVA_OPT="${JAVA_OPT} -DnacosSync.home=${BASE_DIR}"
|
||||||
|
|
||||||
JAVA_MAJOR_VERSION=$($JAVA -version 2>&1 | sed -E -n 's/.* version "([0-9]*).*$/\1/p')
|
JAVA_MAJOR_VERSION=$($JAVA -version 2>&1 | sed -E -n 's/.* version "([0-9]*).*$/\1/p')
|
||||||
if [[ "$JAVA_MAJOR_VERSION" -ge "9" ]] ; then
|
if [[ "$JAVA_MAJOR_VERSION" -ge "9" ]] ; then
|
||||||
JAVA_OPT="${JAVA_OPT} -Xlog:gc*:file=${BASE_DIR}/logs/nacossync_gc.log:time,tags:filecount=10,filesize=102400"
|
JAVA_OPT="${JAVA_OPT} -Xlog:gc*:file=${BASE_DIR}/logs/nacos-sync-gc.log:time,tags:filecount=10,filesize=102400"
|
||||||
else
|
else
|
||||||
JAVA_OPT="${JAVA_OPT} -Xloggc:${BASE_DIR}/logs/nacossync_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M"
|
JAVA_OPT="${JAVA_OPT} -Xloggc:${BASE_DIR}/logs/nacos-sync-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
JAVA_OPT="${JAVA_OPT} -jar ${BASE_DIR}/nacosSync-server.jar"
|
JAVA_OPT="${JAVA_OPT} -jar ${BASE_DIR}/nacos-sync-server.jar"
|
||||||
JAVA_OPT="${JAVA_OPT} --logging.config=${BASE_DIR}/conf/logback-spring.xml"
|
JAVA_OPT="${JAVA_OPT} --logging.config=${BASE_DIR}/conf/logback-spring.xml"
|
||||||
|
|
||||||
echo "=============JAVA_HOME:"$JAVA_HOME
|
echo "JAVA_HOME:"$JAVA_HOME
|
||||||
echo "=============BASE_DIR:"$BASE_DIR
|
echo "BASE_DIR:"$BASE_DIR
|
||||||
echo "=============JAVA:"$JAVA
|
echo "JAVA:"$JAVA
|
||||||
|
|
||||||
|
|
||||||
if [ ! -d "${BASE_DIR}/logs" ]; then
|
if [ ! -d "${BASE_DIR}/logs" ]; then
|
||||||
|
@ -78,9 +78,9 @@ usage(){
|
||||||
|
|
||||||
start(){
|
start(){
|
||||||
|
|
||||||
echo "$JAVA ${JAVA_OPT}" > ${BASE_DIR}/logs/nacossync_start.out 2>&1 &
|
echo "$JAVA ${JAVA_OPT}" > ${BASE_DIR}/logs/nacos-sync-start.out 2>&1 &
|
||||||
nohup $JAVA ${JAVA_OPT} >> ${BASE_DIR}/logs/nacossync_start.out 2>&1 &
|
nohup $JAVA ${JAVA_OPT} >> ${BASE_DIR}/logs/nacos-sync-start.out 2>&1 &
|
||||||
echo "nacossync is starting,you can check the ${BASE_DIR}/logs/nacossync_start.out"
|
echo "nacos-sync is starting,you can check the ${BASE_DIR}/logs/nacos-sync-start.out"
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
@ -5,14 +5,14 @@
|
||||||
<parent>
|
<parent>
|
||||||
<artifactId>nacossync-parent</artifactId>
|
<artifactId>nacossync-parent</artifactId>
|
||||||
<groupId>com.alibaba.nacossync</groupId>
|
<groupId>com.alibaba.nacossync</groupId>
|
||||||
<version>0.4.4</version>
|
<version>${revision}</version>
|
||||||
</parent>
|
</parent>
|
||||||
<modelVersion>4.0.0</modelVersion>
|
<modelVersion>4.0.0</modelVersion>
|
||||||
<packaging>pom</packaging>
|
<packaging>pom</packaging>
|
||||||
<artifactId>nacossync-distribution</artifactId>
|
<artifactId>nacossync-distribution</artifactId>
|
||||||
|
|
||||||
<build>
|
<build>
|
||||||
<finalName>nacosSync.${parent.version}</finalName>
|
<finalName>nacos-sync-${project.version}</finalName>
|
||||||
<plugins>
|
<plugins>
|
||||||
<plugin>
|
<plugin>
|
||||||
<artifactId>maven-assembly-plugin</artifactId>
|
<artifactId>maven-assembly-plugin</artifactId>
|
||||||
|
|
|
@ -17,7 +17,7 @@
|
||||||
<id>bin</id>
|
<id>bin</id>
|
||||||
<includeBaseDirectory>true</includeBaseDirectory>
|
<includeBaseDirectory>true</includeBaseDirectory>
|
||||||
<!-- file name after unzip -->
|
<!-- file name after unzip -->
|
||||||
<baseDirectory>nacosSync</baseDirectory>
|
<baseDirectory>nacos-sync</baseDirectory>
|
||||||
<formats>
|
<formats>
|
||||||
<format>dir</format>
|
<format>dir</format>
|
||||||
<format>tar.gz</format>
|
<format>tar.gz</format>
|
||||||
|
@ -60,9 +60,9 @@
|
||||||
<destName>NOTICE</destName>
|
<destName>NOTICE</destName>
|
||||||
</file>
|
</file>
|
||||||
<file>
|
<file>
|
||||||
<source>../nacossync-worker/target/nacosSync-server.${parent.version}.jar</source>
|
<source>../nacossync-worker/target/nacos-sync-server-${project.version}.jar</source>
|
||||||
<outputDirectory></outputDirectory>
|
<outputDirectory></outputDirectory>
|
||||||
<destName>nacosSync-server.jar</destName>
|
<destName>nacos-sync-server.jar</destName>
|
||||||
</file>
|
</file>
|
||||||
</files>
|
</files>
|
||||||
</assembly>
|
</assembly>
|
||||||
|
|
|
@ -17,7 +17,7 @@
|
||||||
<parent>
|
<parent>
|
||||||
<artifactId>nacossync-parent</artifactId>
|
<artifactId>nacossync-parent</artifactId>
|
||||||
<groupId>com.alibaba.nacossync</groupId>
|
<groupId>com.alibaba.nacossync</groupId>
|
||||||
<version>0.4.4</version>
|
<version>${revision}</version>
|
||||||
<relativePath>../pom.xml</relativePath>
|
<relativePath>../pom.xml</relativePath>
|
||||||
</parent>
|
</parent>
|
||||||
<modelVersion>4.0.0</modelVersion>
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
@ -25,27 +25,19 @@
|
||||||
|
|
||||||
<artifactId>nacossync-test</artifactId>
|
<artifactId>nacossync-test</artifactId>
|
||||||
|
|
||||||
<properties>
|
|
||||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
|
||||||
</properties>
|
|
||||||
<dependencies>
|
<dependencies>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>io.projectreactor</groupId>
|
<groupId>io.projectreactor</groupId>
|
||||||
<artifactId>reactor-core</artifactId>
|
<artifactId>reactor-core</artifactId>
|
||||||
|
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>com.alibaba.nacossync</groupId>
|
<groupId>${project.groupId}</groupId>
|
||||||
<artifactId>nacossync-worker</artifactId>
|
<artifactId>nacossync-worker</artifactId>
|
||||||
<version>0.4.4</version>
|
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>spring-boot-test</artifactId>
|
<artifactId>spring-boot-test</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.springframework</groupId>
|
<groupId>org.springframework</groupId>
|
||||||
<artifactId>spring-test</artifactId>
|
<artifactId>spring-test</artifactId>
|
||||||
|
@ -57,10 +49,8 @@
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>com.ning</groupId>
|
<groupId>com.ning</groupId>
|
||||||
<artifactId>async-http-client</artifactId>
|
<artifactId>async-http-client</artifactId>
|
||||||
<version>1.7.17</version>
|
|
||||||
<scope>test</scope>
|
<scope>test</scope>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
</dependencies>
|
</dependencies>
|
||||||
|
|
||||||
<build>
|
<build>
|
||||||
|
@ -68,7 +58,6 @@
|
||||||
<plugin>
|
<plugin>
|
||||||
<groupId>org.apache.maven.plugins</groupId>
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
<artifactId>maven-surefire-plugin</artifactId>
|
<artifactId>maven-surefire-plugin</artifactId>
|
||||||
<version>2.9</version>
|
|
||||||
<configuration>
|
<configuration>
|
||||||
<parallel>methods</parallel>
|
<parallel>methods</parallel>
|
||||||
<useUnlimitedThreads>true</useUnlimitedThreads>
|
<useUnlimitedThreads>true</useUnlimitedThreads>
|
||||||
|
@ -76,6 +65,4 @@
|
||||||
</plugin>
|
</plugin>
|
||||||
</plugins>
|
</plugins>
|
||||||
</build>
|
</build>
|
||||||
|
|
||||||
|
|
||||||
</project>
|
</project>
|
|
@ -16,29 +16,11 @@
|
||||||
<parent>
|
<parent>
|
||||||
<artifactId>nacossync-parent</artifactId>
|
<artifactId>nacossync-parent</artifactId>
|
||||||
<groupId>com.alibaba.nacossync</groupId>
|
<groupId>com.alibaba.nacossync</groupId>
|
||||||
<version>0.4.4</version>
|
<version>${revision}</version>
|
||||||
</parent>
|
</parent>
|
||||||
<modelVersion>4.0.0</modelVersion>
|
<modelVersion>4.0.0</modelVersion>
|
||||||
<artifactId>nacossync-worker</artifactId>
|
<artifactId>nacossync-worker</artifactId>
|
||||||
<version>0.4.4</version>
|
|
||||||
<properties>
|
|
||||||
<zookeeper.version>3.4.9</zookeeper.version>
|
|
||||||
<curator.version>4.1.0</curator.version>
|
|
||||||
<cloud.version>Finchley.SR2</cloud.version>
|
|
||||||
<mockito.version>1.10.19</mockito.version>
|
|
||||||
<nacos.client.verison>1.3.1</nacos.client.verison>
|
|
||||||
</properties>
|
|
||||||
<dependencyManagement>
|
|
||||||
<dependencies>
|
|
||||||
<dependency>
|
|
||||||
<groupId>org.springframework.cloud</groupId>
|
|
||||||
<artifactId>spring-cloud-dependencies</artifactId>
|
|
||||||
<version>${cloud.version}</version>
|
|
||||||
<type>pom</type>
|
|
||||||
<scope>import</scope>
|
|
||||||
</dependency>
|
|
||||||
</dependencies>
|
|
||||||
</dependencyManagement>
|
|
||||||
<dependencies>
|
<dependencies>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
@ -52,10 +34,6 @@
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>spring-boot-starter-web</artifactId>
|
<artifactId>spring-boot-starter-web</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
|
||||||
<groupId>org.springframework.boot</groupId>
|
|
||||||
<artifactId>spring-boot-starter-logging</artifactId>
|
|
||||||
</dependency>
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>junit</groupId>
|
<groupId>junit</groupId>
|
||||||
<artifactId>junit</artifactId>
|
<artifactId>junit</artifactId>
|
||||||
|
@ -64,12 +42,12 @@
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.mockito</groupId>
|
<groupId>org.mockito</groupId>
|
||||||
<artifactId>mockito-all</artifactId>
|
<artifactId>mockito-all</artifactId>
|
||||||
<version>${mockito.version}</version>
|
|
||||||
<scope>test</scope>
|
<scope>test</scope>
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.projectlombok</groupId>
|
<groupId>org.projectlombok</groupId>
|
||||||
<artifactId>lombok</artifactId>
|
<artifactId>lombok</artifactId>
|
||||||
|
<scope>provided</scope>
|
||||||
</dependency>
|
</dependency>
|
||||||
<!-- 默认使用HikariCP连接池 -->
|
<!-- 默认使用HikariCP连接池 -->
|
||||||
<dependency>
|
<dependency>
|
||||||
|
@ -82,43 +60,36 @@
|
||||||
</dependency>
|
</dependency>
|
||||||
<!-- mysql -->
|
<!-- mysql -->
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>mysql</groupId>
|
<groupId>com.mysql</groupId>
|
||||||
<artifactId>mysql-connector-java</artifactId>
|
<artifactId>mysql-connector-j</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>com.alibaba</groupId>
|
<groupId>org.springdoc</groupId>
|
||||||
<artifactId>fastjson</artifactId>
|
<artifactId>springdoc-openapi-ui</artifactId>
|
||||||
<version>1.2.47</version>
|
|
||||||
</dependency>
|
|
||||||
<!-- swagger2 -->
|
|
||||||
<dependency>
|
|
||||||
<groupId>io.springfox</groupId>
|
|
||||||
<artifactId>springfox-swagger2</artifactId>
|
|
||||||
<version>2.6.1</version>
|
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>io.springfox</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>springfox-swagger-ui</artifactId>
|
<artifactId>spring-boot-starter-test</artifactId>
|
||||||
<version>2.6.1</version>
|
|
||||||
</dependency>
|
</dependency>
|
||||||
<!--nacos-->
|
<!--nacos-->
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>com.alibaba.nacos</groupId>
|
<groupId>com.alibaba.nacos</groupId>
|
||||||
<artifactId>nacos-client</artifactId>
|
<artifactId>nacos-client</artifactId>
|
||||||
<version>${nacos.client.verison}</version>
|
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<!--zookeeper-->
|
<!--zookeeper-->
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.zookeeper</groupId>
|
<groupId>org.apache.zookeeper</groupId>
|
||||||
<artifactId>zookeeper</artifactId>
|
<artifactId>zookeeper</artifactId>
|
||||||
<version>${zookeeper.version}</version>
|
<exclusions>
|
||||||
|
<exclusion>
|
||||||
|
<artifactId>slf4j-log4j12</artifactId>
|
||||||
|
<groupId>org.slf4j</groupId>
|
||||||
|
</exclusion>
|
||||||
|
</exclusions>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.curator</groupId>
|
<groupId>org.apache.curator</groupId>
|
||||||
<artifactId>curator-recipes</artifactId>
|
<artifactId>curator-recipes</artifactId>
|
||||||
<version>${curator.version}</version>
|
|
||||||
<exclusions>
|
<exclusions>
|
||||||
<exclusion>
|
<exclusion>
|
||||||
<groupId>org.apache.zookeeper</groupId>
|
<groupId>org.apache.zookeeper</groupId>
|
||||||
|
@ -126,22 +97,42 @@
|
||||||
</exclusion>
|
</exclusion>
|
||||||
</exclusions>
|
</exclusions>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.apache.curator</groupId>
|
||||||
|
<artifactId>curator-framework</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.apache.curator</groupId>
|
||||||
|
<artifactId>curator-client</artifactId>
|
||||||
|
<exclusions>
|
||||||
|
<exclusion>
|
||||||
|
<artifactId>slf4j-api</artifactId>
|
||||||
|
<groupId>org.slf4j</groupId>
|
||||||
|
</exclusion>
|
||||||
|
</exclusions>
|
||||||
|
</dependency>
|
||||||
<!-- eureka -->
|
<!-- eureka -->
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.springframework.cloud</groupId>
|
<groupId>org.springframework.cloud</groupId>
|
||||||
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
|
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<!-- consul -->
|
<!-- consul -->
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>com.ecwid.consul</groupId>
|
<groupId>com.ecwid.consul</groupId>
|
||||||
<artifactId>consul-api</artifactId>
|
<artifactId>consul-api</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.apache.commons</groupId>
|
||||||
|
<artifactId>commons-lang3</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>com.google.guava</groupId>
|
||||||
|
<artifactId>guava</artifactId>
|
||||||
|
</dependency>
|
||||||
</dependencies>
|
</dependencies>
|
||||||
|
|
||||||
<build>
|
<build>
|
||||||
<finalName>nacosSync-server.${parent.version}</finalName>
|
<finalName>nacos-sync-server-${project.version}</finalName>
|
||||||
<plugins>
|
<plugins>
|
||||||
<plugin>
|
<plugin>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
@ -170,6 +161,22 @@
|
||||||
</archive>
|
</archive>
|
||||||
</configuration>
|
</configuration>
|
||||||
</plugin>
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-compiler-plugin</artifactId>
|
||||||
|
<configuration>
|
||||||
|
<source>17</source>
|
||||||
|
<target>17</target>
|
||||||
|
<annotationProcessorPaths>
|
||||||
|
<path>
|
||||||
|
<groupId>org.projectlombok</groupId>
|
||||||
|
<artifactId>lombok</artifactId>
|
||||||
|
<version>1.18.34</version>
|
||||||
|
</path>
|
||||||
|
</annotationProcessorPaths>
|
||||||
|
</configuration>
|
||||||
|
|
||||||
|
</plugin>
|
||||||
</plugins>
|
</plugins>
|
||||||
</build>
|
</build>
|
||||||
</project>
|
</project>
|
||||||
|
|
|
@ -14,24 +14,32 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync;
|
package com.alibaba.nacossync;
|
||||||
|
|
||||||
|
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||||
import org.springframework.boot.SpringApplication;
|
import org.springframework.boot.SpringApplication;
|
||||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||||
|
|
||||||
import org.springframework.cloud.netflix.eureka.EurekaClientAutoConfiguration;
|
import org.springframework.cloud.netflix.eureka.EurekaClientAutoConfiguration;
|
||||||
import springfox.documentation.swagger2.annotations.EnableSwagger2;
|
import org.springframework.context.ConfigurableApplicationContext;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: SkyWalkerMain.java, v 0.1 2018-09-24 PM12:42 NacosSync Exp $$
|
* @version $Id: SkyWalkerMain.java, v 0.1 2018-09-24 PM12:42 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
@EnableSwagger2
|
|
||||||
@SpringBootApplication(exclude = EurekaClientAutoConfiguration.class)
|
@SpringBootApplication(exclude = EurekaClientAutoConfiguration.class)
|
||||||
public class NacosSyncMain {
|
public class NacosSyncMain {
|
||||||
|
|
||||||
public static void main(String[] args) {
|
public static void main(String[] args) {
|
||||||
|
|
||||||
SpringApplication.run(NacosSyncMain.class, args);
|
ConfigurableApplicationContext context = SpringApplication.run(NacosSyncMain.class, args);
|
||||||
|
|
||||||
|
// Register shutdown callback using Spring Boot's context lifecycle
|
||||||
|
context.registerShutdownHook();
|
||||||
|
context.addApplicationListener(event -> {
|
||||||
|
if (event instanceof org.springframework.context.event.ContextClosedEvent) {
|
||||||
|
BatchTaskExecutor.shutdown();
|
||||||
|
}
|
||||||
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,6 +14,7 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.api;
|
package com.alibaba.nacossync.api;
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
|
@ -53,9 +54,9 @@ public class ClusterApi {
|
||||||
|
|
||||||
private final ClusterListQueryProcessor clusterListQueryProcessor;
|
private final ClusterListQueryProcessor clusterListQueryProcessor;
|
||||||
|
|
||||||
public ClusterApi(
|
public ClusterApi(ClusterAddProcessor clusterAddProcessor, ClusterDeleteProcessor clusterDeleteProcessor,
|
||||||
ClusterAddProcessor clusterAddProcessor, ClusterDeleteProcessor clusterDeleteProcessor,
|
ClusterDetailQueryProcessor clusterDetailQueryProcessor,
|
||||||
ClusterDetailQueryProcessor clusterDetailQueryProcessor, ClusterListQueryProcessor clusterListQueryProcessor) {
|
ClusterListQueryProcessor clusterListQueryProcessor) {
|
||||||
this.clusterAddProcessor = clusterAddProcessor;
|
this.clusterAddProcessor = clusterAddProcessor;
|
||||||
this.clusterDeleteProcessor = clusterDeleteProcessor;
|
this.clusterDeleteProcessor = clusterDeleteProcessor;
|
||||||
this.clusterDetailQueryProcessor = clusterDetailQueryProcessor;
|
this.clusterDetailQueryProcessor = clusterDetailQueryProcessor;
|
||||||
|
@ -65,8 +66,7 @@ public class ClusterApi {
|
||||||
@RequestMapping(path = "/v1/cluster/list", method = RequestMethod.GET)
|
@RequestMapping(path = "/v1/cluster/list", method = RequestMethod.GET)
|
||||||
public ClusterListQueryResult clusters(ClusterListQueryRequest clusterListQueryRequest) {
|
public ClusterListQueryResult clusters(ClusterListQueryRequest clusterListQueryRequest) {
|
||||||
|
|
||||||
return SkyWalkerTemplate.run(clusterListQueryProcessor, clusterListQueryRequest,
|
return SkyWalkerTemplate.run(clusterListQueryProcessor, clusterListQueryRequest, new ClusterListQueryResult());
|
||||||
new ClusterListQueryResult());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@RequestMapping(path = "/v1/cluster/detail", method = RequestMethod.GET)
|
@RequestMapping(path = "/v1/cluster/detail", method = RequestMethod.GET)
|
||||||
|
@ -79,16 +79,14 @@ public class ClusterApi {
|
||||||
@RequestMapping(path = "/v1/cluster/delete", method = RequestMethod.DELETE)
|
@RequestMapping(path = "/v1/cluster/delete", method = RequestMethod.DELETE)
|
||||||
public ClusterDeleteResult deleteCluster(ClusterDeleteRequest clusterDeleteRequest) {
|
public ClusterDeleteResult deleteCluster(ClusterDeleteRequest clusterDeleteRequest) {
|
||||||
|
|
||||||
return SkyWalkerTemplate.run(clusterDeleteProcessor, clusterDeleteRequest,
|
return SkyWalkerTemplate.run(clusterDeleteProcessor, clusterDeleteRequest, new ClusterDeleteResult());
|
||||||
new ClusterDeleteResult());
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@RequestMapping(path = "/v1/cluster/add", method = RequestMethod.POST)
|
@RequestMapping(path = "/v1/cluster/add", method = RequestMethod.POST)
|
||||||
public ClusterAddResult clusterAdd(@RequestBody ClusterAddRequest clusterAddRequest) {
|
public ClusterAddResult clusterAdd(@RequestBody ClusterAddRequest clusterAddRequest) {
|
||||||
|
|
||||||
return SkyWalkerTemplate
|
return SkyWalkerTemplate.run(clusterAddProcessor, clusterAddRequest, new ClusterAddResult());
|
||||||
.run(clusterAddProcessor, clusterAddRequest, new ClusterAddResult());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@RequestMapping(path = "/v1/cluster/types", method = RequestMethod.GET)
|
@RequestMapping(path = "/v1/cluster/types", method = RequestMethod.GET)
|
||||||
|
|
|
@ -14,6 +14,7 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.api;
|
package com.alibaba.nacossync.api;
|
||||||
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
@ -43,27 +44,29 @@ import com.alibaba.nacossync.template.processor.ConfigQueryProcessor;
|
||||||
@RestController
|
@RestController
|
||||||
public class SystemConfigApi {
|
public class SystemConfigApi {
|
||||||
|
|
||||||
@Autowired
|
private final ConfigQueryProcessor configQueryProcessor;
|
||||||
private ConfigQueryProcessor configQueryProcessor;
|
|
||||||
|
|
||||||
@Autowired
|
private final ConfigDeleteProcessor configDeleteProcessor;
|
||||||
private ConfigDeleteProcessor configDeleteProcessor;
|
|
||||||
|
|
||||||
@Autowired
|
private final ConfigAddProcessor configAddProcessor;
|
||||||
private ConfigAddProcessor configAddProcessor;
|
|
||||||
|
public SystemConfigApi(ConfigQueryProcessor configQueryProcessor, ConfigDeleteProcessor configDeleteProcessor,
|
||||||
|
ConfigAddProcessor configAddProcessor) {
|
||||||
|
this.configQueryProcessor = configQueryProcessor;
|
||||||
|
this.configDeleteProcessor = configDeleteProcessor;
|
||||||
|
this.configAddProcessor = configAddProcessor;
|
||||||
|
}
|
||||||
|
|
||||||
@RequestMapping(path = "/v1/systemconfig/list", method = RequestMethod.GET)
|
@RequestMapping(path = "/v1/systemconfig/list", method = RequestMethod.GET)
|
||||||
public ConfigQueryResult tasks(ConfigQueryRequest configQueryRequest) {
|
public ConfigQueryResult tasks(ConfigQueryRequest configQueryRequest) {
|
||||||
|
|
||||||
return SkyWalkerTemplate.run(configQueryProcessor, configQueryRequest,
|
return SkyWalkerTemplate.run(configQueryProcessor, configQueryRequest, new ConfigQueryResult());
|
||||||
new ConfigQueryResult());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@RequestMapping(path = "/v1/systemconfig/delete", method = RequestMethod.DELETE)
|
@RequestMapping(path = "/v1/systemconfig/delete", method = RequestMethod.DELETE)
|
||||||
public ConfigDeleteResult deleteTask(@RequestBody ConfigDeleteRequest configDeleteRequest) {
|
public ConfigDeleteResult deleteTask(@RequestBody ConfigDeleteRequest configDeleteRequest) {
|
||||||
|
|
||||||
return SkyWalkerTemplate.run(configDeleteProcessor, configDeleteRequest,
|
return SkyWalkerTemplate.run(configDeleteProcessor, configDeleteRequest, new ConfigDeleteResult());
|
||||||
new ConfigDeleteResult());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@RequestMapping(path = "/v1/systemconfig/add", method = RequestMethod.POST)
|
@RequestMapping(path = "/v1/systemconfig/add", method = RequestMethod.POST)
|
||||||
|
|
|
@ -10,8 +10,10 @@
|
||||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
* specific language governing permissions and limitations under the License.
|
* specific language governing permissions and limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.api;
|
package com.alibaba.nacossync.api;
|
||||||
|
|
||||||
|
import com.alibaba.nacossync.pojo.request.TaskAddAllRequest;
|
||||||
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
||||||
import com.alibaba.nacossync.pojo.request.TaskDeleteInBatchRequest;
|
import com.alibaba.nacossync.pojo.request.TaskDeleteInBatchRequest;
|
||||||
import com.alibaba.nacossync.pojo.request.TaskDeleteRequest;
|
import com.alibaba.nacossync.pojo.request.TaskDeleteRequest;
|
||||||
|
@ -23,6 +25,7 @@ import com.alibaba.nacossync.pojo.result.TaskAddResult;
|
||||||
import com.alibaba.nacossync.pojo.result.TaskDetailQueryResult;
|
import com.alibaba.nacossync.pojo.result.TaskDetailQueryResult;
|
||||||
import com.alibaba.nacossync.pojo.result.TaskListQueryResult;
|
import com.alibaba.nacossync.pojo.result.TaskListQueryResult;
|
||||||
import com.alibaba.nacossync.template.SkyWalkerTemplate;
|
import com.alibaba.nacossync.template.SkyWalkerTemplate;
|
||||||
|
import com.alibaba.nacossync.template.processor.TaskAddAllProcessor;
|
||||||
import com.alibaba.nacossync.template.processor.TaskAddProcessor;
|
import com.alibaba.nacossync.template.processor.TaskAddProcessor;
|
||||||
import com.alibaba.nacossync.template.processor.TaskDeleteInBatchProcessor;
|
import com.alibaba.nacossync.template.processor.TaskDeleteInBatchProcessor;
|
||||||
import com.alibaba.nacossync.template.processor.TaskDeleteProcessor;
|
import com.alibaba.nacossync.template.processor.TaskDeleteProcessor;
|
||||||
|
@ -47,6 +50,8 @@ public class TaskApi {
|
||||||
|
|
||||||
private final TaskAddProcessor taskAddProcessor;
|
private final TaskAddProcessor taskAddProcessor;
|
||||||
|
|
||||||
|
private final TaskAddAllProcessor taskAddAllProcessor;
|
||||||
|
|
||||||
private final TaskDeleteProcessor taskDeleteProcessor;
|
private final TaskDeleteProcessor taskDeleteProcessor;
|
||||||
|
|
||||||
private final TaskDeleteInBatchProcessor taskDeleteInBatchProcessor;
|
private final TaskDeleteInBatchProcessor taskDeleteInBatchProcessor;
|
||||||
|
@ -56,10 +61,12 @@ public class TaskApi {
|
||||||
private final TaskDetailProcessor taskDetailProcessor;
|
private final TaskDetailProcessor taskDetailProcessor;
|
||||||
|
|
||||||
public TaskApi(TaskUpdateProcessor taskUpdateProcessor, TaskAddProcessor taskAddProcessor,
|
public TaskApi(TaskUpdateProcessor taskUpdateProcessor, TaskAddProcessor taskAddProcessor,
|
||||||
TaskDeleteProcessor taskDeleteProcessor, TaskDeleteInBatchProcessor taskDeleteInBatchProcessor,
|
TaskAddAllProcessor taskAddAllProcessor, TaskDeleteProcessor taskDeleteProcessor,
|
||||||
TaskListQueryProcessor taskListQueryProcessor, TaskDetailProcessor taskDetailProcessor) {
|
TaskDeleteInBatchProcessor taskDeleteInBatchProcessor, TaskListQueryProcessor taskListQueryProcessor,
|
||||||
|
TaskDetailProcessor taskDetailProcessor) {
|
||||||
this.taskUpdateProcessor = taskUpdateProcessor;
|
this.taskUpdateProcessor = taskUpdateProcessor;
|
||||||
this.taskAddProcessor = taskAddProcessor;
|
this.taskAddProcessor = taskAddProcessor;
|
||||||
|
this.taskAddAllProcessor = taskAddAllProcessor;
|
||||||
this.taskDeleteProcessor = taskDeleteProcessor;
|
this.taskDeleteProcessor = taskDeleteProcessor;
|
||||||
this.taskDeleteInBatchProcessor = taskDeleteInBatchProcessor;
|
this.taskDeleteInBatchProcessor = taskDeleteInBatchProcessor;
|
||||||
this.taskListQueryProcessor = taskListQueryProcessor;
|
this.taskListQueryProcessor = taskListQueryProcessor;
|
||||||
|
@ -85,9 +92,9 @@ public class TaskApi {
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author yongchao9
|
|
||||||
* @param taskBatchDeleteRequest
|
* @param taskBatchDeleteRequest
|
||||||
* @return
|
* @return
|
||||||
|
* @author yongchao9
|
||||||
*/
|
*/
|
||||||
@RequestMapping(path = "/v1/task/deleteInBatch", method = RequestMethod.DELETE)
|
@RequestMapping(path = "/v1/task/deleteInBatch", method = RequestMethod.DELETE)
|
||||||
public BaseResult batchDeleteTask(TaskDeleteInBatchRequest taskBatchDeleteRequest) {
|
public BaseResult batchDeleteTask(TaskDeleteInBatchRequest taskBatchDeleteRequest) {
|
||||||
|
@ -100,6 +107,18 @@ public class TaskApi {
|
||||||
return SkyWalkerTemplate.run(taskAddProcessor, addTaskRequest, new TaskAddResult());
|
return SkyWalkerTemplate.run(taskAddProcessor, addTaskRequest, new TaskAddResult());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* TODO 目前仅支持 Nacos 为源的同步类型,待完善更多类型支持.
|
||||||
|
* <p>
|
||||||
|
* 支持从 sourceCluster 获取所有 service,然后生成同步到 destCluster 的任务。
|
||||||
|
* </p>
|
||||||
|
*/
|
||||||
|
@RequestMapping(path = "/v1/task/addAll", method = RequestMethod.POST)
|
||||||
|
public BaseResult taskAddAll(@RequestBody TaskAddAllRequest addAllRequest) {
|
||||||
|
|
||||||
|
return SkyWalkerTemplate.run(taskAddAllProcessor, addAllRequest, new TaskAddResult());
|
||||||
|
}
|
||||||
|
|
||||||
@RequestMapping(path = "/v1/task/update", method = RequestMethod.POST)
|
@RequestMapping(path = "/v1/task/update", method = RequestMethod.POST)
|
||||||
public BaseResult updateTask(@RequestBody TaskUpdateRequest taskUpdateRequest) {
|
public BaseResult updateTask(@RequestBody TaskUpdateRequest taskUpdateRequest) {
|
||||||
|
|
||||||
|
|
|
@ -16,8 +16,7 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.cache;
|
package com.alibaba.nacossync.cache;
|
||||||
|
|
||||||
import com.alibaba.fastjson.JSONObject;
|
|
||||||
import com.alibaba.fastjson.TypeReference;
|
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||||
|
@ -25,15 +24,17 @@ import com.alibaba.nacossync.pojo.FinishedTask;
|
||||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||||
import java.util.List;
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
import java.util.Map;
|
import lombok.SneakyThrows;
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
|
||||||
import org.jboss.netty.util.internal.ThreadLocalRandom;
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
import org.springframework.util.CollectionUtils;
|
import org.springframework.util.CollectionUtils;
|
||||||
import org.springframework.util.StringUtils;
|
import org.springframework.util.StringUtils;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
import java.util.concurrent.ThreadLocalRandom;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: SkyWalkerCacheServices.java, v 0.1 2018-09-27 AM2:47 NacosSync Exp $$
|
* @version $Id: SkyWalkerCacheServices.java, v 0.1 2018-09-27 AM2:47 NacosSync Exp $$
|
||||||
|
@ -41,23 +42,28 @@ import org.springframework.util.StringUtils;
|
||||||
@Service
|
@Service
|
||||||
public class SkyWalkerCacheServices {
|
public class SkyWalkerCacheServices {
|
||||||
|
|
||||||
@Autowired
|
private static final Map<String, FinishedTask> FINISHED_TASK_MAP = new ConcurrentHashMap<>();
|
||||||
private ClusterAccessService clusterAccessService;
|
|
||||||
|
|
||||||
private static Map<String, FinishedTask> finishedTaskMap = new ConcurrentHashMap<>();
|
private final ClusterAccessService clusterAccessService;
|
||||||
|
|
||||||
|
private final ObjectMapper objectMapper;
|
||||||
|
|
||||||
|
public SkyWalkerCacheServices(ClusterAccessService clusterAccessService, ObjectMapper objectMapper) {
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
this.objectMapper = objectMapper;
|
||||||
|
}
|
||||||
|
|
||||||
public String getClusterConnectKey(String clusterId) {
|
public String getClusterConnectKey(String clusterId) {
|
||||||
List<String> allClusterConnectKey = getAllClusterConnectKey(clusterId);
|
List<String> allClusterConnectKey = getAllClusterConnectKey(clusterId);
|
||||||
|
|
||||||
return allClusterConnectKey.get(ThreadLocalRandom.current().nextInt(allClusterConnectKey.size()));
|
return allClusterConnectKey.get(ThreadLocalRandom.current().nextInt(allClusterConnectKey.size()));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@SneakyThrows
|
||||||
public List<String> getAllClusterConnectKey(String clusterId) {
|
public List<String> getAllClusterConnectKey(String clusterId) {
|
||||||
ClusterDO clusterDO = clusterAccessService.findByClusterId(clusterId);
|
ClusterDO clusterDO = clusterAccessService.findByClusterId(clusterId);
|
||||||
|
|
||||||
List<String> connectKeyList = JSONObject.parseObject(clusterDO.getConnectKeyList(),
|
List<String> connectKeyList = objectMapper.readerForListOf(String.class)
|
||||||
new TypeReference<List<String>>() {
|
.readValue(clusterDO.getConnectKeyList());
|
||||||
});
|
|
||||||
|
|
||||||
if (CollectionUtils.isEmpty(connectKeyList)) {
|
if (CollectionUtils.isEmpty(connectKeyList)) {
|
||||||
throw new SkyWalkerException("getClusterConnectKey empty, clusterId:" + clusterId);
|
throw new SkyWalkerException("getClusterConnectKey empty, clusterId:" + clusterId);
|
||||||
|
@ -79,23 +85,32 @@ public class SkyWalkerCacheServices {
|
||||||
FinishedTask finishedTask = new FinishedTask();
|
FinishedTask finishedTask = new FinishedTask();
|
||||||
finishedTask.setOperationId(operationId);
|
finishedTask.setOperationId(operationId);
|
||||||
|
|
||||||
finishedTaskMap.put(operationId, finishedTask);
|
FINISHED_TASK_MAP.put(operationId, finishedTask);
|
||||||
}
|
}
|
||||||
|
|
||||||
public FinishedTask getFinishedTask(TaskDO taskDO) {
|
public FinishedTask getFinishedTask(TaskDO taskDO) {
|
||||||
|
|
||||||
String operationId = SkyWalkerUtil.getOperationId(taskDO);
|
String operationId = SkyWalkerUtil.getOperationId(taskDO);
|
||||||
|
|
||||||
if (StringUtils.isEmpty(operationId)) {
|
if (!StringUtils.hasLength(operationId)) {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
return finishedTaskMap.get(operationId);
|
return FINISHED_TASK_MAP.get(operationId);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
public void removeFinishedTask(String operationId) {
|
||||||
|
if (!StringUtils.hasLength(operationId)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
FINISHED_TASK_MAP.remove(operationId);
|
||||||
}
|
}
|
||||||
|
|
||||||
public Map<String, FinishedTask> getFinishedTaskMap() {
|
public Map<String, FinishedTask> getFinishedTaskMap() {
|
||||||
|
|
||||||
return finishedTaskMap;
|
return FINISHED_TASK_MAP;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -12,6 +12,8 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.constant;
|
package com.alibaba.nacossync.constant;
|
||||||
|
|
||||||
|
import lombok.Getter;
|
||||||
|
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
|
||||||
|
@ -19,6 +21,7 @@ import java.util.List;
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: ClusterTypeEnum.java, v 0.1 2018-09-25 下午4:38 NacosSync Exp $$
|
* @version $Id: ClusterTypeEnum.java, v 0.1 2018-09-25 下午4:38 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
|
@Getter
|
||||||
public enum ClusterTypeEnum {
|
public enum ClusterTypeEnum {
|
||||||
|
|
||||||
CS("CS", "configserver集群"),
|
CS("CS", "configserver集群"),
|
||||||
|
@ -32,18 +35,16 @@ public enum ClusterTypeEnum {
|
||||||
ZK("ZK", "zookeeper集群");
|
ZK("ZK", "zookeeper集群");
|
||||||
|
|
||||||
|
|
||||||
private String code;
|
private final String code;
|
||||||
|
|
||||||
private String desc;
|
|
||||||
|
|
||||||
ClusterTypeEnum(String code, String desc) {
|
ClusterTypeEnum(String code, String desc) {
|
||||||
this.code = code;
|
this.code = code;
|
||||||
this.desc = desc;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public static List<String> getClusterTypeCodes() {
|
public static List<String> getClusterTypeCodes() {
|
||||||
|
|
||||||
List<String> list = new ArrayList<String>();
|
List<String> list = new ArrayList<>();
|
||||||
|
|
||||||
for (ClusterTypeEnum clusterTypeEnum : ClusterTypeEnum.values()) {
|
for (ClusterTypeEnum clusterTypeEnum : ClusterTypeEnum.values()) {
|
||||||
list.add(clusterTypeEnum.getCode());
|
list.add(clusterTypeEnum.getCode());
|
||||||
|
@ -51,41 +52,6 @@ public enum ClusterTypeEnum {
|
||||||
return list;
|
return list;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Getter method for property <tt>code</tt>.
|
|
||||||
*
|
|
||||||
* @return property value of code
|
|
||||||
*/
|
|
||||||
public String getCode() {
|
|
||||||
return code;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Setter method for property <tt>code </tt>.
|
|
||||||
*
|
|
||||||
* @param code value to be assigned to property code
|
|
||||||
*/
|
|
||||||
public void setCode(String code) {
|
|
||||||
this.code = code;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Getter method for property <tt>desc</tt>.
|
|
||||||
*
|
|
||||||
* @return property value of desc
|
|
||||||
*/
|
|
||||||
public String getDesc() {
|
|
||||||
return desc;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Setter method for property <tt>desc </tt>.
|
|
||||||
*
|
|
||||||
* @param desc value to be assigned to property desc
|
|
||||||
*/
|
|
||||||
public void setDesc(String desc) {
|
|
||||||
this.desc = desc;
|
|
||||||
}
|
|
||||||
|
|
||||||
public static boolean contains(String clusterType) {
|
public static boolean contains(String clusterType) {
|
||||||
|
|
||||||
|
|
|
@ -3,10 +3,13 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.constant;
|
package com.alibaba.nacossync.constant;
|
||||||
|
|
||||||
|
import lombok.Getter;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: MetricsStatisticsType.java, v 0.1 2019年02月28日 下午2:17 NacosSync Exp $
|
* @version $Id: MetricsStatisticsType.java, v 0.1 2019年02月28日 下午2:17 NacosSync Exp $
|
||||||
*/
|
*/
|
||||||
|
@Getter
|
||||||
public enum MetricsStatisticsType {
|
public enum MetricsStatisticsType {
|
||||||
|
|
||||||
CACHE_SIZE("nacosSync.finished.taskMap.cacheSize", "任务执行完成缓存列表数"),
|
CACHE_SIZE("nacosSync.finished.taskMap.cacheSize", "任务执行完成缓存列表数"),
|
||||||
|
@ -28,15 +31,10 @@ public enum MetricsStatisticsType {
|
||||||
/**
|
/**
|
||||||
* metricsName
|
* metricsName
|
||||||
*/
|
*/
|
||||||
private String metricsName;
|
private final String metricsName;
|
||||||
private String desc;
|
|
||||||
|
|
||||||
MetricsStatisticsType(String code, String desc) {
|
MetricsStatisticsType(String code, String desc) {
|
||||||
this.metricsName = code;
|
this.metricsName = code;
|
||||||
this.desc = desc;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getMetricsName() {
|
|
||||||
return metricsName;
|
|
||||||
}
|
|
||||||
}
|
}
|
|
@ -16,46 +16,23 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.constant;
|
package com.alibaba.nacossync.constant;
|
||||||
|
|
||||||
|
import lombok.Getter;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: ResultCodeEnum.java, v 0.1 2018-09-25 PM4:38 NacosSync Exp $$
|
* @version $Id: ResultCodeEnum.java, v 0.1 2018-09-25 PM4:38 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
|
@Getter
|
||||||
public enum ResultCodeEnum {
|
public enum ResultCodeEnum {
|
||||||
|
|
||||||
SUCCESS("SUCCESS", "请求成功", "请求成功"),
|
SUCCESS("SUCCESS", "请求成功", "请求成功"),
|
||||||
SYSTEM_ERROR("SYSTEM_ERROR", "系统异常", "系统异常");
|
SYSTEM_ERROR("SYSTEM_ERROR", "系统异常", "系统异常");
|
||||||
|
|
||||||
private String code;
|
private final String code;
|
||||||
private String errorMessage;
|
|
||||||
private String detail;
|
|
||||||
|
|
||||||
ResultCodeEnum(String code, String errorMessage, String detail) {
|
ResultCodeEnum(String code, String errorMessage, String detail) {
|
||||||
this.code = code;
|
this.code = code;
|
||||||
this.errorMessage = errorMessage;
|
|
||||||
this.detail = detail;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getCode() {
|
|
||||||
return code;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setCode(String code) {
|
|
||||||
this.code = code;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getErrorMessage() {
|
|
||||||
return errorMessage;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setErrorMessage(String errorMessage) {
|
|
||||||
this.errorMessage = errorMessage;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getDetail() {
|
|
||||||
return detail;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setDetail(String detail) {
|
|
||||||
this.detail = detail;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -24,11 +24,18 @@ public class SkyWalkerConstants {
|
||||||
|
|
||||||
public static final String UNDERLINE = "_";
|
public static final String UNDERLINE = "_";
|
||||||
|
|
||||||
public static final String DEST_CLUSTERID_KEY = "destClusterId";
|
public static final String DEST_CLUSTER_ID_KEY = "destClusterId";
|
||||||
public static final String GROUP_NAME = "groupName";
|
public static final String GROUP_NAME = "groupName";
|
||||||
public static final String SYNC_SOURCE_KEY = "syncSource";
|
public static final String SYNC_SOURCE_KEY = "syncSource";
|
||||||
public static final String SOURCE_CLUSTERID_KEY = "sourceClusterId";
|
public static final String SOURCE_CLUSTER_ID_KEY = "sourceClusterId";
|
||||||
public static final String MANAGEMENT_PORT_KEY="management.port";
|
public static final String MANAGEMENT_PORT_KEY="management.port";
|
||||||
public static final String MANAGEMENT_CONTEXT_PATH_KEY="management.context-path";
|
public static final String MANAGEMENT_CONTEXT_PATH_KEY="management.context-path";
|
||||||
|
|
||||||
|
public static final String SERVICE_NAME_PARAM="serviceNameParam";
|
||||||
|
public static final String GROUP_NAME_PARAM="groupNameParam";
|
||||||
|
public static final String PAGE_NO="pageNo";
|
||||||
|
public static final String PAGE_SIZE="pageSize";
|
||||||
|
public static final String SYNC_INSTANCE_TAG="sync.instance.tag";
|
||||||
|
public static final String NACOS_ALL_SERVICE_NAME = "ALL";
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -16,6 +16,8 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.constant;
|
package com.alibaba.nacossync.constant;
|
||||||
|
|
||||||
|
import lombok.Getter;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: TaskStatusEnum.java, v 0.1 2018-09-26 上午2:38 NacosSync Exp $$
|
* @version $Id: TaskStatusEnum.java, v 0.1 2018-09-26 上午2:38 NacosSync Exp $$
|
||||||
|
@ -31,49 +33,16 @@ public enum TaskStatusEnum {
|
||||||
*/
|
*/
|
||||||
DELETE("DELETE", "任务需要被删除");
|
DELETE("DELETE", "任务需要被删除");
|
||||||
|
|
||||||
private String code;
|
|
||||||
private String desc;
|
@Getter
|
||||||
|
private final String code;
|
||||||
|
private final String desc;
|
||||||
|
|
||||||
TaskStatusEnum(String code, String desc) {
|
TaskStatusEnum(String code, String desc) {
|
||||||
this.code = code;
|
this.code = code;
|
||||||
this.desc = desc;
|
this.desc = desc;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Getter method for property <tt>code</tt>.
|
|
||||||
*
|
|
||||||
* @return property value of code
|
|
||||||
*/
|
|
||||||
public String getCode() {
|
|
||||||
return code;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Setter method for property <tt>code </tt>.
|
|
||||||
*
|
|
||||||
* @param code value to be assigned to property code
|
|
||||||
*/
|
|
||||||
public void setCode(String code) {
|
|
||||||
this.code = code;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Getter method for property <tt>desc</tt>.
|
|
||||||
*
|
|
||||||
* @return property value of desc
|
|
||||||
*/
|
|
||||||
public String getDesc() {
|
|
||||||
return desc;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Setter method for property <tt>desc </tt>.
|
|
||||||
*
|
|
||||||
* @param desc value to be assigned to property desc
|
|
||||||
*/
|
|
||||||
public void setDesc(String desc) {
|
|
||||||
this.desc = desc;
|
|
||||||
}
|
|
||||||
|
|
||||||
public static boolean contains(String code) {
|
public static boolean contains(String code) {
|
||||||
|
|
||||||
|
|
|
@ -16,10 +16,10 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.dao;
|
package com.alibaba.nacossync.dao;
|
||||||
|
|
||||||
|
import com.alibaba.nacossync.dao.repository.ClusterRepository;
|
||||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||||
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.data.domain.Page;
|
import org.springframework.data.domain.Page;
|
||||||
import org.springframework.data.domain.PageRequest;
|
import org.springframework.data.domain.PageRequest;
|
||||||
import org.springframework.data.domain.Pageable;
|
import org.springframework.data.domain.Pageable;
|
||||||
|
@ -27,9 +27,6 @@ import org.springframework.data.domain.Sort;
|
||||||
import org.springframework.data.jpa.domain.Specification;
|
import org.springframework.data.jpa.domain.Specification;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.repository.ClusterRepository;
|
|
||||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
|
||||||
|
|
||||||
import javax.persistence.criteria.CriteriaBuilder;
|
import javax.persistence.criteria.CriteriaBuilder;
|
||||||
import javax.persistence.criteria.Predicate;
|
import javax.persistence.criteria.Predicate;
|
||||||
import javax.persistence.criteria.Root;
|
import javax.persistence.criteria.Root;
|
||||||
|
@ -44,8 +41,11 @@ import java.util.List;
|
||||||
@Slf4j
|
@Slf4j
|
||||||
public class ClusterAccessService implements PageQueryService<ClusterDO> {
|
public class ClusterAccessService implements PageQueryService<ClusterDO> {
|
||||||
|
|
||||||
@Autowired
|
private final ClusterRepository clusterRepository;
|
||||||
private ClusterRepository clusterRepository;
|
|
||||||
|
public ClusterAccessService(ClusterRepository clusterRepository) {
|
||||||
|
this.clusterRepository = clusterRepository;
|
||||||
|
}
|
||||||
|
|
||||||
public ClusterDO insert(ClusterDO clusterDO) {
|
public ClusterDO insert(ClusterDO clusterDO) {
|
||||||
|
|
||||||
|
@ -103,4 +103,12 @@ public class ClusterAccessService implements PageQueryService<ClusterDO> {
|
||||||
predicates.add(criteriaBuilder.like(root.get("clusterName"), "%" + queryCondition.getServiceName() + "%"));
|
predicates.add(criteriaBuilder.like(root.get("clusterName"), "%" + queryCondition.getServiceName() + "%"));
|
||||||
return predicates;
|
return predicates;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public int findClusterLevel(String sourceClusterId){
|
||||||
|
ClusterDO clusterDO = clusterRepository.findByClusterId(sourceClusterId);
|
||||||
|
if (clusterDO != null) {
|
||||||
|
return clusterDO.getClusterLevel();
|
||||||
|
}
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,10 +14,13 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.dao;
|
package com.alibaba.nacossync.dao;
|
||||||
|
|
||||||
|
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||||
|
import com.alibaba.nacossync.dao.repository.TaskRepository;
|
||||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import org.springframework.data.domain.Page;
|
import org.springframework.data.domain.Page;
|
||||||
import org.springframework.data.domain.PageRequest;
|
import org.springframework.data.domain.PageRequest;
|
||||||
import org.springframework.data.domain.Pageable;
|
import org.springframework.data.domain.Pageable;
|
||||||
|
@ -25,9 +28,6 @@ import org.springframework.data.domain.Sort;
|
||||||
import org.springframework.data.jpa.domain.Specification;
|
import org.springframework.data.jpa.domain.Specification;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.repository.TaskRepository;
|
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
|
||||||
|
|
||||||
import javax.persistence.criteria.CriteriaBuilder;
|
import javax.persistence.criteria.CriteriaBuilder;
|
||||||
import javax.persistence.criteria.Predicate;
|
import javax.persistence.criteria.Predicate;
|
||||||
import javax.persistence.criteria.Root;
|
import javax.persistence.criteria.Root;
|
||||||
|
@ -41,8 +41,11 @@ import java.util.List;
|
||||||
@Service
|
@Service
|
||||||
public class TaskAccessService implements PageQueryService<TaskDO> {
|
public class TaskAccessService implements PageQueryService<TaskDO> {
|
||||||
|
|
||||||
@Autowired
|
private final TaskRepository taskRepository;
|
||||||
private TaskRepository taskRepository;
|
|
||||||
|
public TaskAccessService(TaskRepository taskRepository) {
|
||||||
|
this.taskRepository = taskRepository;
|
||||||
|
}
|
||||||
|
|
||||||
public TaskDO findByTaskId(String taskId) {
|
public TaskDO findByTaskId(String taskId) {
|
||||||
|
|
||||||
|
@ -55,12 +58,13 @@ public class TaskAccessService implements PageQueryService<TaskDO> {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* batch delete tasks by taskIds
|
* batch delete tasks by taskIds
|
||||||
* @author yongchao9
|
*
|
||||||
* @param taskIds
|
* @param taskIds
|
||||||
|
* @author yongchao9
|
||||||
*/
|
*/
|
||||||
public void deleteTaskInBatch(List<String> taskIds) {
|
public void deleteTaskInBatch(List<String> taskIds) {
|
||||||
List<TaskDO> tds=taskRepository.findAllByTaskIdIn(taskIds);
|
List<TaskDO> tds = taskRepository.findAllByTaskIdIn(taskIds);
|
||||||
taskRepository.deleteInBatch(tds);
|
taskRepository.deleteAllInBatch(tds);
|
||||||
}
|
}
|
||||||
|
|
||||||
public Iterable<TaskDO> findAll() {
|
public Iterable<TaskDO> findAll() {
|
||||||
|
@ -74,12 +78,17 @@ public class TaskAccessService implements PageQueryService<TaskDO> {
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public int countByDestClusterIdOrSourceClusterId(String destClusterId, String sourceClusterId) {
|
||||||
|
return taskRepository.countByDestClusterIdOrSourceClusterId(destClusterId, sourceClusterId);
|
||||||
|
}
|
||||||
|
|
||||||
private Predicate getPredicate(CriteriaBuilder criteriaBuilder, List<Predicate> predicates) {
|
private Predicate getPredicate(CriteriaBuilder criteriaBuilder, List<Predicate> predicates) {
|
||||||
Predicate[] p = new Predicate[predicates.size()];
|
Predicate[] p = new Predicate[predicates.size()];
|
||||||
return criteriaBuilder.and(predicates.toArray(p));
|
return criteriaBuilder.and(predicates.toArray(p));
|
||||||
}
|
}
|
||||||
|
|
||||||
private List<Predicate> getPredicates(Root<TaskDO> root, CriteriaBuilder criteriaBuilder, QueryCondition queryCondition) {
|
private List<Predicate> getPredicates(Root<TaskDO> root, CriteriaBuilder criteriaBuilder,
|
||||||
|
QueryCondition queryCondition) {
|
||||||
|
|
||||||
List<Predicate> predicates = new ArrayList<>();
|
List<Predicate> predicates = new ArrayList<>();
|
||||||
predicates.add(criteriaBuilder.like(root.get("serviceName"), "%" + queryCondition.getServiceName() + "%"));
|
predicates.add(criteriaBuilder.like(root.get("serviceName"), "%" + queryCondition.getServiceName() + "%"));
|
||||||
|
@ -104,15 +113,21 @@ public class TaskAccessService implements PageQueryService<TaskDO> {
|
||||||
}
|
}
|
||||||
|
|
||||||
private Page<TaskDO> getTaskDOS(QueryCondition queryCondition, Pageable pageable) {
|
private Page<TaskDO> getTaskDOS(QueryCondition queryCondition, Pageable pageable) {
|
||||||
return taskRepository.findAll(
|
return taskRepository.findAll((Specification<TaskDO>) (root, criteriaQuery, criteriaBuilder) -> {
|
||||||
(Specification<TaskDO>) (root, criteriaQuery, criteriaBuilder) -> {
|
|
||||||
|
|
||||||
List<Predicate> predicates = getPredicates(root,
|
List<Predicate> predicates = getPredicates(root, criteriaBuilder, queryCondition);
|
||||||
criteriaBuilder, queryCondition);
|
|
||||||
|
|
||||||
return getPredicate(criteriaBuilder, predicates);
|
return getPredicate(criteriaBuilder, predicates);
|
||||||
|
|
||||||
}, pageable);
|
}, pageable);
|
||||||
|
}
|
||||||
|
|
||||||
|
public List<TaskDO> findAllByServiceNameEqualAll() {
|
||||||
|
return taskRepository.findAllByServiceNameEqualsIgnoreCase(SkyWalkerConstants.NACOS_ALL_SERVICE_NAME);
|
||||||
|
}
|
||||||
|
|
||||||
|
public List<TaskDO> findAllByServiceNameNotEqualAll() {
|
||||||
|
return taskRepository.findAllByServiceNameNotIgnoreCase(SkyWalkerConstants.NACOS_ALL_SERVICE_NAME);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -16,13 +16,13 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.dao.repository;
|
package com.alibaba.nacossync.dao.repository;
|
||||||
|
|
||||||
import javax.transaction.Transactional;
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
|
|
||||||
import org.springframework.data.jpa.repository.JpaRepository;
|
import org.springframework.data.jpa.repository.JpaRepository;
|
||||||
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
|
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
|
||||||
import org.springframework.data.repository.CrudRepository;
|
import org.springframework.data.repository.CrudRepository;
|
||||||
|
|
||||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
import javax.transaction.Transactional;
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
|
@ -34,6 +34,6 @@ public interface ClusterRepository extends CrudRepository<ClusterDO, Integer>, J
|
||||||
ClusterDO findByClusterId(String clusterId);
|
ClusterDO findByClusterId(String clusterId);
|
||||||
|
|
||||||
@Transactional
|
@Transactional
|
||||||
int deleteByClusterId(String clusterId);
|
void deleteByClusterId(String clusterId);
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -16,15 +16,13 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.dao.repository;
|
package com.alibaba.nacossync.dao.repository;
|
||||||
|
|
||||||
import java.util.List;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
|
||||||
import javax.transaction.Transactional;
|
|
||||||
|
|
||||||
import org.springframework.data.jpa.repository.JpaRepository;
|
import org.springframework.data.jpa.repository.JpaRepository;
|
||||||
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
|
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
|
||||||
import org.springframework.data.repository.CrudRepository;
|
import org.springframework.data.repository.CrudRepository;
|
||||||
|
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import javax.transaction.Transactional;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
|
@ -36,10 +34,15 @@ public interface TaskRepository extends CrudRepository<TaskDO, Integer>, JpaRepo
|
||||||
TaskDO findByTaskId(String taskId);
|
TaskDO findByTaskId(String taskId);
|
||||||
|
|
||||||
@Transactional
|
@Transactional
|
||||||
int deleteByTaskId(String taskId);
|
void deleteByTaskId(String taskId);
|
||||||
|
|
||||||
List<TaskDO> findAllByTaskIdIn(List<String> taskIds);
|
List<TaskDO> findAllByTaskIdIn(List<String> taskIds);
|
||||||
|
/**
|
||||||
|
* query service is all,use ns leven sync data
|
||||||
|
*/
|
||||||
|
List<TaskDO> findAllByServiceNameEqualsIgnoreCase(String serviceName);
|
||||||
|
List<TaskDO> findAllByServiceNameNotIgnoreCase(String serviceName);
|
||||||
|
|
||||||
List<TaskDO> getAllByWorkerIp(String workerIp);
|
int countByDestClusterIdOrSourceClusterId(String destClusterId,String sourceClusterId);
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -0,0 +1,13 @@
|
||||||
|
package com.alibaba.nacossync.event;
|
||||||
|
|
||||||
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
public class DeleteAllSubTaskEvent {
|
||||||
|
public DeleteAllSubTaskEvent(TaskDO taskDO) {
|
||||||
|
this.taskDO = taskDO;
|
||||||
|
}
|
||||||
|
|
||||||
|
private final TaskDO taskDO;
|
||||||
|
}
|
|
@ -14,23 +14,22 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.event.listener;
|
package com.alibaba.nacossync.event.listener;
|
||||||
|
|
||||||
import javax.annotation.PostConstruct;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||||
|
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||||
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||||
import com.alibaba.nacossync.event.SyncTaskEvent;
|
import com.alibaba.nacossync.event.SyncTaskEvent;
|
||||||
import com.alibaba.nacossync.extension.SyncManagerService;
|
import com.alibaba.nacossync.extension.SyncManagerService;
|
||||||
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
|
import com.google.common.base.Stopwatch;
|
||||||
import com.google.common.eventbus.EventBus;
|
import com.google.common.eventbus.EventBus;
|
||||||
import com.google.common.eventbus.Subscribe;
|
import com.google.common.eventbus.Subscribe;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import javax.annotation.PostConstruct;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
|
@ -40,17 +39,21 @@ import com.google.common.eventbus.Subscribe;
|
||||||
@Service
|
@Service
|
||||||
public class EventListener {
|
public class EventListener {
|
||||||
|
|
||||||
@Autowired
|
private final MetricsManager metricsManager;
|
||||||
private MetricsManager metricsManager;
|
|
||||||
|
|
||||||
@Autowired
|
private final SyncManagerService syncManagerService;
|
||||||
private SyncManagerService syncManagerService;
|
|
||||||
|
|
||||||
@Autowired
|
private final EventBus eventBus;
|
||||||
private EventBus eventBus;
|
|
||||||
|
|
||||||
@Autowired
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
|
||||||
|
public EventListener(MetricsManager metricsManager, SyncManagerService syncManagerService, EventBus eventBus,
|
||||||
|
SkyWalkerCacheServices skyWalkerCacheServices) {
|
||||||
|
this.metricsManager = metricsManager;
|
||||||
|
this.syncManagerService = syncManagerService;
|
||||||
|
this.eventBus = eventBus;
|
||||||
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
|
}
|
||||||
|
|
||||||
@PostConstruct
|
@PostConstruct
|
||||||
public void register() {
|
public void register() {
|
||||||
|
@ -58,31 +61,36 @@ public class EventListener {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Subscribe
|
@Subscribe
|
||||||
public void listenerSyncTaskEvent(SyncTaskEvent syncTaskEvent) {
|
public void sync(SyncTaskEvent syncTaskEvent) {
|
||||||
|
|
||||||
try {
|
try {
|
||||||
long start = System.currentTimeMillis();
|
|
||||||
syncManagerService.sync(syncTaskEvent.getTaskDO());
|
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||||
skyWalkerCacheServices.addFinishedTask(syncTaskEvent.getTaskDO());
|
if (syncManagerService.sync(syncTaskEvent.getTaskDO(), null)) {
|
||||||
metricsManager.record(MetricsStatisticsType.SYNC_TASK_RT, System.currentTimeMillis() - start);
|
skyWalkerCacheServices.addFinishedTask(syncTaskEvent.getTaskDO());
|
||||||
|
metricsManager.record(MetricsStatisticsType.SYNC_TASK_RT, stopwatch.elapsed().toMillis());
|
||||||
|
} else {
|
||||||
|
log.warn("syncTaskEvent process error");
|
||||||
|
}
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.warn("listenerSyncTaskEvent process error", e);
|
log.warn("syncTaskEvent process error", e);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@Subscribe
|
@Subscribe
|
||||||
public void listenerDeleteTaskEvent(DeleteTaskEvent deleteTaskEvent) {
|
public void delete(DeleteTaskEvent deleteTaskEvent) {
|
||||||
|
|
||||||
try {
|
try {
|
||||||
long start = System.currentTimeMillis();
|
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||||
syncManagerService.delete(deleteTaskEvent.getTaskDO());
|
if (syncManagerService.delete(deleteTaskEvent.getTaskDO())) {
|
||||||
skyWalkerCacheServices.addFinishedTask(deleteTaskEvent.getTaskDO());
|
skyWalkerCacheServices.removeFinishedTask(deleteTaskEvent.getTaskDO().getOperationId());
|
||||||
metricsManager.record(MetricsStatisticsType.DELETE_TASK_RT, System.currentTimeMillis() - start);
|
metricsManager.record(MetricsStatisticsType.DELETE_TASK_RT, stopwatch.elapsed().toMillis());
|
||||||
|
} else {
|
||||||
|
log.warn("deleteTaskEvent delete failure");
|
||||||
|
}
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.warn("listenerDeleteTaskEvent process error", e);
|
log.warn("deleteTaskEvent delete failure.", e);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -12,14 +12,12 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.extension;
|
package com.alibaba.nacossync.extension;
|
||||||
|
|
||||||
import static com.alibaba.nacossync.util.SkyWalkerUtil.generateSyncKey;
|
|
||||||
|
|
||||||
import com.alibaba.nacos.api.exception.NacosException;
|
import com.alibaba.nacos.api.exception.NacosException;
|
||||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
import com.alibaba.nacossync.util.StringUtils;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.BeansException;
|
import org.springframework.beans.BeansException;
|
||||||
import org.springframework.beans.factory.InitializingBean;
|
import org.springframework.beans.factory.InitializingBean;
|
||||||
|
@ -27,6 +25,10 @@ import org.springframework.context.ApplicationContext;
|
||||||
import org.springframework.context.ApplicationContextAware;
|
import org.springframework.context.ApplicationContextAware;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
|
||||||
|
import static com.alibaba.nacossync.util.SkyWalkerUtil.generateSyncKey;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: SyncManagerService.java, v 0.1 2018-09-25 PM5:17 NacosSync Exp $$
|
* @version $Id: SyncManagerService.java, v 0.1 2018-09-25 PM5:17 NacosSync Exp $$
|
||||||
|
@ -37,7 +39,7 @@ public class SyncManagerService implements InitializingBean, ApplicationContextA
|
||||||
|
|
||||||
protected final SkyWalkerCacheServices skyWalkerCacheServices;
|
protected final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
|
|
||||||
private ConcurrentHashMap<String, SyncService> syncServiceMap = new ConcurrentHashMap<String, SyncService>();
|
private final ConcurrentHashMap<String, SyncService> syncServiceMap = new ConcurrentHashMap<String, SyncService>();
|
||||||
|
|
||||||
private ApplicationContext applicationContext;
|
private ApplicationContext applicationContext;
|
||||||
|
|
||||||
|
@ -52,19 +54,19 @@ public class SyncManagerService implements InitializingBean, ApplicationContextA
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public boolean sync(TaskDO taskDO) {
|
public boolean sync(TaskDO taskDO, Integer index) {
|
||||||
|
|
||||||
return getSyncService(taskDO.getSourceClusterId(), taskDO.getDestClusterId()).sync(taskDO);
|
return getSyncService(taskDO.getSourceClusterId(), taskDO.getDestClusterId()).sync(taskDO, index);
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void afterPropertiesSet() {
|
public void afterPropertiesSet() {
|
||||||
this.applicationContext.getBeansOfType(SyncService.class).forEach((key, value) -> {
|
this.applicationContext.getBeansWithAnnotation(NacosSyncService.class).forEach((key, value) -> {
|
||||||
NacosSyncService nacosSyncService = value.getClass().getAnnotation(NacosSyncService.class);
|
NacosSyncService nacosSyncService = value.getClass().getAnnotation(NacosSyncService.class);
|
||||||
ClusterTypeEnum sourceCluster = nacosSyncService.sourceCluster();
|
ClusterTypeEnum sourceCluster = nacosSyncService.sourceCluster();
|
||||||
ClusterTypeEnum destinationCluster = nacosSyncService.destinationCluster();
|
ClusterTypeEnum destinationCluster = nacosSyncService.destinationCluster();
|
||||||
syncServiceMap.put(generateSyncKey(sourceCluster, destinationCluster), value);
|
syncServiceMap.put(generateSyncKey(sourceCluster, destinationCluster), (SyncService) value);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -74,7 +76,9 @@ public class SyncManagerService implements InitializingBean, ApplicationContextA
|
||||||
}
|
}
|
||||||
|
|
||||||
public SyncService getSyncService(String sourceClusterId, String destClusterId) {
|
public SyncService getSyncService(String sourceClusterId, String destClusterId) {
|
||||||
|
if (StringUtils.isEmpty(sourceClusterId) || StringUtils.isEmpty(destClusterId)) {
|
||||||
|
throw new IllegalArgumentException("Source cluster id and destination cluster id must not be null or empty");
|
||||||
|
}
|
||||||
ClusterTypeEnum sourceClusterType = this.skyWalkerCacheServices.getClusterType(sourceClusterId);
|
ClusterTypeEnum sourceClusterType = this.skyWalkerCacheServices.getClusterType(sourceClusterId);
|
||||||
ClusterTypeEnum destClusterType = this.skyWalkerCacheServices.getClusterType(destClusterId);
|
ClusterTypeEnum destClusterType = this.skyWalkerCacheServices.getClusterType(destClusterId);
|
||||||
|
|
||||||
|
|
|
@ -14,9 +14,10 @@ package com.alibaba.nacossync.extension;
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import java.util.Map;
|
|
||||||
import org.apache.commons.lang3.StringUtils;
|
import org.apache.commons.lang3.StringUtils;
|
||||||
|
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: SyncManagerService.java, v 0.1 2018-09-25 下午5:17 NacosSync Exp $$
|
* @version $Id: SyncManagerService.java, v 0.1 2018-09-25 下午5:17 NacosSync Exp $$
|
||||||
|
@ -32,18 +33,20 @@ public interface SyncService {
|
||||||
boolean delete(TaskDO taskDO);
|
boolean delete(TaskDO taskDO);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* execute sync
|
|
||||||
*
|
*
|
||||||
* @param taskDO
|
* @param taskDO
|
||||||
|
* @param index
|
||||||
* @return
|
* @return
|
||||||
*/
|
*/
|
||||||
boolean sync(TaskDO taskDO);
|
boolean sync(TaskDO taskDO, Integer index);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Determines that the current instance data is from another source cluster
|
* Determines that the current instance data is from another source cluster
|
||||||
*/
|
*/
|
||||||
default boolean needSync(Map<String, String> sourceMetaData) {
|
default boolean needSync(Map<String, String> sourceMetaData) {
|
||||||
return StringUtils.isBlank(sourceMetaData.get(SkyWalkerConstants.SOURCE_CLUSTERID_KEY));
|
boolean syncTag = StringUtils.isBlank(sourceMetaData.get(SkyWalkerConstants.SYNC_INSTANCE_TAG));
|
||||||
|
boolean blank = StringUtils.isBlank(sourceMetaData.get(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY));
|
||||||
|
return syncTag && blank;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -51,7 +54,7 @@ public interface SyncService {
|
||||||
* cluster ID of the task
|
* cluster ID of the task
|
||||||
*/
|
*/
|
||||||
default boolean needDelete(Map<String, String> destMetaData, TaskDO taskDO) {
|
default boolean needDelete(Map<String, String> destMetaData, TaskDO taskDO) {
|
||||||
return StringUtils.equals(destMetaData.get(SkyWalkerConstants.SOURCE_CLUSTERID_KEY),
|
return StringUtils.equals(destMetaData.get(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY),
|
||||||
taskDO.getSourceClusterId());
|
taskDO.getSourceClusterId());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
@ -1,15 +0,0 @@
|
||||||
package com.alibaba.nacossync.extension.client;
|
|
||||||
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
public class InstanceQueryModel {
|
|
||||||
|
|
||||||
private String sourceClusterId;
|
|
||||||
private String destClusterId;
|
|
||||||
private String serviceName;
|
|
||||||
private String groupName;
|
|
||||||
private String version;
|
|
||||||
private int pageNo;
|
|
||||||
private int pageSize;
|
|
||||||
}
|
|
|
@ -1,11 +0,0 @@
|
||||||
package com.alibaba.nacossync.extension.client;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
public interface SyncQueryClient {
|
|
||||||
|
|
||||||
|
|
||||||
List<TaskModel> getAllInstance(InstanceQueryModel instanceQueryModel);
|
|
||||||
|
|
||||||
}
|
|
|
@ -1,54 +0,0 @@
|
||||||
package com.alibaba.nacossync.extension.client.impl;
|
|
||||||
|
|
||||||
import com.alibaba.nacos.api.exception.NacosException;
|
|
||||||
import com.alibaba.nacos.api.naming.NamingService;
|
|
||||||
import com.alibaba.nacos.api.naming.pojo.ListView;
|
|
||||||
import com.alibaba.nacossync.extension.client.InstanceQueryModel;
|
|
||||||
import com.alibaba.nacossync.extension.client.SyncQueryClient;
|
|
||||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
|
||||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
|
||||||
import java.util.Collections;
|
|
||||||
import java.util.List;
|
|
||||||
import java.util.stream.Collectors;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
@Service
|
|
||||||
@Slf4j
|
|
||||||
public class NacosSyncQueryClientImpl implements SyncQueryClient {
|
|
||||||
|
|
||||||
private final NacosServerHolder nacosServerHolder;
|
|
||||||
|
|
||||||
public NacosSyncQueryClientImpl(
|
|
||||||
NacosServerHolder nacosServerHolder) {
|
|
||||||
this.nacosServerHolder = nacosServerHolder;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public List<TaskModel> getAllInstance(InstanceQueryModel instanceQueryModel) {
|
|
||||||
NamingService namingService = nacosServerHolder
|
|
||||||
.get(instanceQueryModel.getSourceClusterId(), instanceQueryModel.getGroupName());
|
|
||||||
try {
|
|
||||||
ListView<String> servicesOfServer = namingService
|
|
||||||
.getServicesOfServer(instanceQueryModel.getPageNo(),
|
|
||||||
instanceQueryModel.getPageSize());
|
|
||||||
return servicesOfServer.getData().stream()
|
|
||||||
.map(serviceName -> buildTaskModel(instanceQueryModel, serviceName))
|
|
||||||
.collect(Collectors.toList());
|
|
||||||
|
|
||||||
} catch (NacosException e) {
|
|
||||||
log.error("When using nacos client failure query tasks", e);
|
|
||||||
return Collections.emptyList();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private TaskModel buildTaskModel(InstanceQueryModel instanceQueryModel, String serviceName) {
|
|
||||||
TaskModel taskModel = new TaskModel();
|
|
||||||
taskModel.setServiceName(serviceName);
|
|
||||||
taskModel.setSourceClusterId(instanceQueryModel.getSourceClusterId());
|
|
||||||
taskModel.setDestClusterId(instanceQueryModel.getDestClusterId());
|
|
||||||
return taskModel;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
}
|
|
|
@ -29,11 +29,11 @@ import java.util.concurrent.TimeUnit;
|
||||||
*/
|
*/
|
||||||
@Slf4j
|
@Slf4j
|
||||||
public class EurekaBeatReactor {
|
public class EurekaBeatReactor {
|
||||||
private ScheduledExecutorService executorService;
|
private final ScheduledExecutorService executorService;
|
||||||
|
|
||||||
private volatile long clientBeatInterval = 5 * 1000;
|
private static final long CLIENT_BEAT_INTERVAL = 5 * 1000;
|
||||||
private final Map<String, InstanceInfo> eurekaBeat = new ConcurrentHashMap<>();
|
private final Map<String, InstanceInfo> eurekaBeat = new ConcurrentHashMap<>();
|
||||||
private EurekaHttpClient eurekaHttpClient;
|
private final EurekaHttpClient eurekaHttpClient;
|
||||||
|
|
||||||
public EurekaBeatReactor(EurekaHttpClient eurekaHttpClient) {
|
public EurekaBeatReactor(EurekaHttpClient eurekaHttpClient) {
|
||||||
this.eurekaHttpClient = eurekaHttpClient;
|
this.eurekaHttpClient = eurekaHttpClient;
|
||||||
|
@ -71,7 +71,7 @@ public class EurekaBeatReactor {
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("[CLIENT-BEAT] Exception while scheduling beat.", e);
|
log.error("[CLIENT-BEAT] Exception while scheduling beat.", e);
|
||||||
} finally {
|
} finally {
|
||||||
executorService.schedule(this, clientBeatInterval, TimeUnit.MILLISECONDS);
|
executorService.schedule(this, CLIENT_BEAT_INTERVAL, TimeUnit.MILLISECONDS);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -26,8 +26,8 @@ import java.util.Objects;
|
||||||
* @date 2019-06-26
|
* @date 2019-06-26
|
||||||
*/
|
*/
|
||||||
public class EurekaNamingService {
|
public class EurekaNamingService {
|
||||||
private EurekaHttpClient eurekaHttpClient;
|
private final EurekaHttpClient eurekaHttpClient;
|
||||||
private EurekaBeatReactor beatReactor;
|
private final EurekaBeatReactor beatReactor;
|
||||||
|
|
||||||
|
|
||||||
public EurekaNamingService(EurekaHttpClient eurekaHttpClient) {
|
public EurekaNamingService(EurekaHttpClient eurekaHttpClient) {
|
||||||
|
|
|
@ -25,7 +25,7 @@ import java.util.function.Consumer;
|
||||||
*/
|
*/
|
||||||
@Service
|
@Service
|
||||||
public class SpecialSyncEventBus {
|
public class SpecialSyncEventBus {
|
||||||
private ConcurrentHashMap<String, SpecialSyncEvent> specialSyncEventRegistry = new ConcurrentHashMap<>();
|
private final ConcurrentHashMap<String, SpecialSyncEvent> specialSyncEventRegistry = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
public void subscribe(TaskDO taskDO, Consumer<TaskDO> syncAction) {
|
public void subscribe(TaskDO taskDO, Consumer<TaskDO> syncAction) {
|
||||||
SpecialSyncEvent specialSyncEvent = new SpecialSyncEvent();
|
SpecialSyncEvent specialSyncEvent = new SpecialSyncEvent();
|
||||||
|
|
|
@ -21,7 +21,6 @@ import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.google.common.eventbus.EventBus;
|
import com.google.common.eventbus.EventBus;
|
||||||
import com.google.common.eventbus.Subscribe;
|
import com.google.common.eventbus.Subscribe;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import javax.annotation.PostConstruct;
|
import javax.annotation.PostConstruct;
|
||||||
|
@ -34,8 +33,11 @@ import java.util.function.Consumer;
|
||||||
@Service
|
@Service
|
||||||
@Slf4j
|
@Slf4j
|
||||||
public class SpecialSyncEventListener {
|
public class SpecialSyncEventListener {
|
||||||
@Autowired
|
private final EventBus eventBus;
|
||||||
private EventBus eventBus;
|
|
||||||
|
public SpecialSyncEventListener(EventBus eventBus) {
|
||||||
|
this.eventBus = eventBus;
|
||||||
|
}
|
||||||
|
|
||||||
@PostConstruct
|
@PostConstruct
|
||||||
public void init() {
|
public void init() {
|
||||||
|
|
|
@ -13,13 +13,10 @@
|
||||||
package com.alibaba.nacossync.extension.holder;
|
package com.alibaba.nacossync.extension.holder;
|
||||||
|
|
||||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||||
import com.google.common.base.Joiner;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.apache.logging.log4j.util.Strings;
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
|
||||||
import java.util.Map;
|
import java.util.Map;
|
||||||
import java.util.Optional;
|
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
import java.util.function.Supplier;
|
import java.util.function.Supplier;
|
||||||
|
|
||||||
|
@ -28,40 +25,36 @@ import java.util.function.Supplier;
|
||||||
* @date 2018-12-24 22:08
|
* @date 2018-12-24 22:08
|
||||||
*/
|
*/
|
||||||
@Slf4j
|
@Slf4j
|
||||||
public abstract class AbstractServerHolderImpl<T> implements Holder {
|
public abstract class AbstractServerHolderImpl<T> implements Holder<T> {
|
||||||
|
|
||||||
private final Map<String, T> serviceMap = new ConcurrentHashMap<>();
|
private final Map<String, T> serviceMap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
|
|
||||||
@Autowired
|
@Autowired
|
||||||
protected SkyWalkerCacheServices skyWalkerCacheServices;
|
protected SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public T get(String clusterId, String namespace) {
|
public T get(String clusterId) {
|
||||||
final String finalNamespace = Optional.ofNullable(namespace).orElse(Strings.EMPTY);
|
|
||||||
String key = Joiner.on("_").join(clusterId, finalNamespace);
|
|
||||||
|
|
||||||
serviceMap.computeIfAbsent(key, clusterKey -> {
|
return serviceMap.computeIfAbsent(clusterId, clusterKey -> {
|
||||||
try {
|
try {
|
||||||
log.info("Starting create cluster server, clusterId={}", clusterId);
|
log.info("Starting create cluster server, clusterId={}", clusterId);
|
||||||
return createServer(clusterId, () -> skyWalkerCacheServices.getClusterConnectKey(clusterId),
|
return createServer(clusterId, () -> skyWalkerCacheServices.getClusterConnectKey(clusterId));
|
||||||
finalNamespace);
|
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error(String.format("clusterId=%s, start server failed", clusterId), e);
|
log.error(String.format("clusterId=%s, start server failed", clusterId), e);
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
return serviceMap.get(key);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Create real cluster client instance
|
* Create real cluster client instance
|
||||||
*
|
*
|
||||||
* @param clusterId cluster id
|
* @param clusterId cluster id
|
||||||
* @param serverAddressSupplier server address
|
* @param serverAddressSupplier server address
|
||||||
* @param namespace name space
|
|
||||||
* @return cluster client instance
|
* @return cluster client instance
|
||||||
*/
|
*/
|
||||||
abstract T createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace)
|
abstract T createServer(String clusterId, Supplier<String> serverAddressSupplier)
|
||||||
throws Exception;
|
throws Exception;
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -30,7 +30,7 @@ public class ConsulServerHolder extends AbstractServerHolderImpl<ConsulClient> {
|
||||||
public static final String HTTP = "http://";
|
public static final String HTTP = "http://";
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
ConsulClient createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace) throws Exception {
|
ConsulClient createServer(String clusterId, Supplier<String> serverAddressSupplier) throws Exception {
|
||||||
String serverAddress = serverAddressSupplier.get();
|
String serverAddress = serverAddressSupplier.get();
|
||||||
serverAddress = serverAddress.startsWith(HTTP) ? serverAddress : HTTP + serverAddress;
|
serverAddress = serverAddress.startsWith(HTTP) ? serverAddress : HTTP + serverAddress;
|
||||||
URL url = new URL(serverAddress);
|
URL url = new URL(serverAddress);
|
||||||
|
|
|
@ -10,6 +10,7 @@
|
||||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
* specific language governing permissions and limitations under the License.
|
* specific language governing permissions and limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.extension.holder;
|
package com.alibaba.nacossync.extension.holder;
|
||||||
|
|
||||||
import com.alibaba.nacossync.extension.eureka.EurekaNamingService;
|
import com.alibaba.nacossync.extension.eureka.EurekaNamingService;
|
||||||
|
@ -30,12 +31,27 @@ import java.util.function.Supplier;
|
||||||
@Service
|
@Service
|
||||||
@Slf4j
|
@Slf4j
|
||||||
public class EurekaServerHolder extends AbstractServerHolderImpl<EurekaNamingService> {
|
public class EurekaServerHolder extends AbstractServerHolderImpl<EurekaNamingService> {
|
||||||
|
|
||||||
|
private static final String HTTP_PREFIX = "http://";
|
||||||
|
|
||||||
|
private static final String HTTPS_PREFIX = "https://";
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
EurekaNamingService createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace) throws Exception {
|
EurekaNamingService createServer(String clusterId, Supplier<String> serverAddressSupplier) throws Exception {
|
||||||
RestTemplateTransportClientFactory restTemplateTransportClientFactory =
|
RestTemplateTransportClientFactory restTemplateTransportClientFactory = new RestTemplateTransportClientFactory();
|
||||||
new RestTemplateTransportClientFactory();
|
EurekaEndpoint eurekaEndpoint = new DefaultEndpoint(addHttpPrefix(serverAddressSupplier.get()));
|
||||||
EurekaEndpoint eurekaEndpoint = new DefaultEndpoint(serverAddressSupplier.get());
|
|
||||||
EurekaHttpClient eurekaHttpClient = restTemplateTransportClientFactory.newClient(eurekaEndpoint);
|
EurekaHttpClient eurekaHttpClient = restTemplateTransportClientFactory.newClient(eurekaEndpoint);
|
||||||
return new EurekaNamingService(eurekaHttpClient);
|
return new EurekaNamingService(eurekaHttpClient);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public String addHttpPrefix(String input) {
|
||||||
|
if (input == null || input.isEmpty()) {
|
||||||
|
return input;
|
||||||
|
}
|
||||||
|
if (!input.startsWith(HTTP_PREFIX) && !input.startsWith(HTTPS_PREFIX)) {
|
||||||
|
input = HTTP_PREFIX + input;
|
||||||
|
}
|
||||||
|
|
||||||
|
return input;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -22,9 +22,8 @@ public interface Holder<T> {
|
||||||
/**
|
/**
|
||||||
* Through the cluster ID and namespace fetch cluster client service
|
* Through the cluster ID and namespace fetch cluster client service
|
||||||
* @param clusterId cluster id
|
* @param clusterId cluster id
|
||||||
* @param namespace name space
|
|
||||||
* @return
|
* @return
|
||||||
* @throws Exception
|
* @throws Exception
|
||||||
*/
|
*/
|
||||||
T get(String clusterId, String namespace) throws Exception;
|
T get(String clusterId) throws Exception;
|
||||||
}
|
}
|
||||||
|
|
|
@ -18,12 +18,14 @@ import com.alibaba.nacos.api.naming.NamingService;
|
||||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
import com.google.common.base.Joiner;
|
import com.google.common.base.Joiner;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.apache.logging.log4j.util.Strings;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Optional;
|
import java.util.Optional;
|
||||||
import java.util.Properties;
|
import java.util.Properties;
|
||||||
import java.util.function.Supplier;
|
import java.util.function.Supplier;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author paderlol
|
* @author paderlol
|
||||||
|
@ -40,7 +42,7 @@ public class NacosServerHolder extends AbstractServerHolderImpl<NamingService> {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
NamingService createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace)
|
NamingService createServer(String clusterId, Supplier<String> serverAddressSupplier)
|
||||||
throws Exception {
|
throws Exception {
|
||||||
List<String> allClusterConnectKey = skyWalkerCacheServices
|
List<String> allClusterConnectKey = skyWalkerCacheServices
|
||||||
.getAllClusterConnectKey(clusterId);
|
.getAllClusterConnectKey(clusterId);
|
||||||
|
@ -48,7 +50,8 @@ public class NacosServerHolder extends AbstractServerHolderImpl<NamingService> {
|
||||||
String serverList = Joiner.on(",").join(allClusterConnectKey);
|
String serverList = Joiner.on(",").join(allClusterConnectKey);
|
||||||
Properties properties = new Properties();
|
Properties properties = new Properties();
|
||||||
properties.setProperty(PropertyKeyConst.SERVER_ADDR, serverList);
|
properties.setProperty(PropertyKeyConst.SERVER_ADDR, serverList);
|
||||||
properties.setProperty(PropertyKeyConst.NAMESPACE, namespace);
|
properties.setProperty(PropertyKeyConst.NAMESPACE, Optional.ofNullable(clusterDO.getNamespace()).orElse(
|
||||||
|
Strings.EMPTY));
|
||||||
Optional.ofNullable(clusterDO.getUserName()).ifPresent(value ->
|
Optional.ofNullable(clusterDO.getUserName()).ifPresent(value ->
|
||||||
properties.setProperty(PropertyKeyConst.USERNAME, value)
|
properties.setProperty(PropertyKeyConst.USERNAME, value)
|
||||||
);
|
);
|
||||||
|
@ -58,4 +61,5 @@ public class NacosServerHolder extends AbstractServerHolderImpl<NamingService> {
|
||||||
);
|
);
|
||||||
return NamingFactory.createNamingService(properties);
|
return NamingFactory.createNamingService(properties);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -32,7 +32,7 @@ public class ZookeeperServerHolder extends AbstractServerHolderImpl<CuratorFrame
|
||||||
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
CuratorFramework createServer(String clusterId, Supplier<String> serverAddressSupplier, String namespace) {
|
CuratorFramework createServer(String clusterId, Supplier<String> serverAddressSupplier) {
|
||||||
List<String> allClusterConnectKey = skyWalkerCacheServices
|
List<String> allClusterConnectKey = skyWalkerCacheServices
|
||||||
.getAllClusterConnectKey(clusterId);
|
.getAllClusterConnectKey(clusterId);
|
||||||
String serverList = Joiner.on(",").join(allClusterConnectKey);
|
String serverList = Joiner.on(",").join(allClusterConnectKey);
|
||||||
|
|
|
@ -0,0 +1,265 @@
|
||||||
|
package com.alibaba.nacossync.extension.impl;
|
||||||
|
|
||||||
|
import com.alibaba.nacos.api.exception.NacosException;
|
||||||
|
import com.alibaba.nacos.api.naming.NamingService;
|
||||||
|
import com.alibaba.nacos.api.naming.listener.EventListener;
|
||||||
|
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
||||||
|
import com.alibaba.nacos.api.naming.pojo.Instance;
|
||||||
|
import com.alibaba.nacos.common.utils.CollectionUtils;
|
||||||
|
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||||
|
import com.alibaba.nacossync.extension.SyncService;
|
||||||
|
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||||
|
import com.google.common.base.Stopwatch;
|
||||||
|
import lombok.Getter;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
|
||||||
|
import javax.annotation.PostConstruct;
|
||||||
|
import javax.annotation.PreDestroy;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.Collection;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.TreeSet;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
import java.util.concurrent.Executors;
|
||||||
|
import java.util.concurrent.ScheduledExecutorService;
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
|
||||||
|
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||||
|
|
||||||
|
@Slf4j
|
||||||
|
public abstract class AbstractNacosSync implements SyncService {
|
||||||
|
|
||||||
|
private final Map<String, EventListener> listenerMap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
|
private final Map<String, Set<String>> sourceInstanceSnapshot = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
|
private final Map<String, Integer> syncTaskTap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
|
private final ConcurrentHashMap<String, TaskDO> allSyncTaskMap = new ConcurrentHashMap<>();
|
||||||
|
private ScheduledExecutorService executorService;
|
||||||
|
@Autowired
|
||||||
|
private MetricsManager metricsManager;
|
||||||
|
|
||||||
|
@Getter
|
||||||
|
@Autowired
|
||||||
|
private NacosServerHolder nacosServerHolder;
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Due to network issues or other reasons, the Nacos Sync synchronization tasks may fail,
|
||||||
|
* resulting in the target cluster's registry missing synchronized instances.
|
||||||
|
* To prevent the target cluster's registry from missing synchronized instances for an extended period,
|
||||||
|
* a fallback worker thread is started every 5 minutes to execute all synchronization tasks.
|
||||||
|
*/
|
||||||
|
@PostConstruct
|
||||||
|
public void afterPropertiesSet() {
|
||||||
|
initializeExecutorService();
|
||||||
|
scheduleSyncTasks();
|
||||||
|
}
|
||||||
|
|
||||||
|
@PreDestroy
|
||||||
|
public void destroy() {
|
||||||
|
if (executorService != null && !executorService.isShutdown()) {
|
||||||
|
executorService.shutdown();
|
||||||
|
try {
|
||||||
|
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||||
|
executorService.shutdownNow();
|
||||||
|
}
|
||||||
|
} catch (InterruptedException e) {
|
||||||
|
executorService.shutdownNow();
|
||||||
|
Thread.currentThread().interrupt();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void initializeExecutorService() {
|
||||||
|
executorService = Executors.newSingleThreadScheduledExecutor(r -> {
|
||||||
|
Thread t = new Thread(r);
|
||||||
|
t.setDaemon(true);
|
||||||
|
t.setName("com.alibaba.nacossync.basic.synctask");
|
||||||
|
return t;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private void scheduleSyncTasks() {
|
||||||
|
executorService.scheduleWithFixedDelay(this::executeSyncTasks, 0, 300, TimeUnit.SECONDS);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void executeSyncTasks() {
|
||||||
|
if (allSyncTaskMap.isEmpty()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
Collection<TaskDO> taskCollections = allSyncTaskMap.values();
|
||||||
|
List<TaskDO> taskDOList = new ArrayList<>(taskCollections);
|
||||||
|
|
||||||
|
if (CollectionUtils.isNotEmpty(taskDOList)) {
|
||||||
|
BatchTaskExecutor.batchOperation(taskDOList, this::executeTask);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void executeTask(TaskDO task) {
|
||||||
|
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||||
|
String taskId = task.getTaskId();
|
||||||
|
try {
|
||||||
|
NamingService sourceNamingService = nacosServerHolder.get(task.getSourceClusterId());
|
||||||
|
doSync(taskId, task, sourceNamingService);
|
||||||
|
} catch (NacosException e) {
|
||||||
|
log.error("sync task from nacos to nacos failed, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
} finally {
|
||||||
|
stopwatch.stop();
|
||||||
|
log.debug("Task execution time for taskId {}: {} ms", taskId, stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public boolean delete(TaskDO taskDO) {
|
||||||
|
String taskId = taskDO.getTaskId();
|
||||||
|
try {
|
||||||
|
|
||||||
|
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||||
|
//移除订阅
|
||||||
|
EventListener listener = listenerMap.remove(taskId);
|
||||||
|
if (listener!= null) {
|
||||||
|
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()), listener);
|
||||||
|
}
|
||||||
|
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
|
listenerMap.remove(taskId));
|
||||||
|
sourceInstanceSnapshot.remove(taskId);
|
||||||
|
allSyncTaskMap.remove(taskId);
|
||||||
|
|
||||||
|
// 删除目标集群中同步的实例列表
|
||||||
|
deregisterInstance(taskDO);
|
||||||
|
}catch (NacosException e) {
|
||||||
|
log.error("Delete task from nacos to specify destination was failed, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
return false;
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public boolean sync(TaskDO taskDO, Integer index) {
|
||||||
|
String taskId = taskDO.getTaskId();
|
||||||
|
try {
|
||||||
|
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||||
|
allSyncTaskMap.put(taskId, taskDO);
|
||||||
|
//防止暂停同步任务后,重新同步/或删除任务以后新建任务不会再接收到新的事件导致不能同步,所以每次订阅事件之前,先全量同步一次任务
|
||||||
|
doSync(taskId, taskDO, sourceNamingService);
|
||||||
|
this.listenerMap.putIfAbsent(taskId, event -> {
|
||||||
|
if (event instanceof NamingEvent) {
|
||||||
|
try {
|
||||||
|
doSync(taskId, taskDO, sourceNamingService);
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("event process fail, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
sourceNamingService.subscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
|
listenerMap.get(taskId));
|
||||||
|
}catch (NacosException e) {
|
||||||
|
log.error("Nacos sync task process fail, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
return false;
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void doSync(String taskId, TaskDO taskDO, NamingService sourceNamingService) throws Exception {
|
||||||
|
if (syncTaskTap.putIfAbsent(taskId, 1) != null) {
|
||||||
|
log.info("任务Id:{}上一个同步任务尚未结束", taskId);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
// 直接从本地保存的serviceInfoMap中取订阅的服务实例
|
||||||
|
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||||
|
// 先删除不存在的
|
||||||
|
this.removeInvalidInstance(taskDO, sourceInstances);
|
||||||
|
|
||||||
|
// 同步实例
|
||||||
|
this.syncNewInstance(taskDO, sourceInstances);
|
||||||
|
} finally {
|
||||||
|
syncTaskTap.remove(taskId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void syncNewInstance(TaskDO taskDO, List<Instance> sourceInstances) throws NacosException {
|
||||||
|
Set<String> latestSyncInstance = new TreeSet<>();
|
||||||
|
//再次添加新实例
|
||||||
|
String taskId = taskDO.getTaskId();
|
||||||
|
Set<String> instanceKeys = sourceInstanceSnapshot.get(taskId);
|
||||||
|
for (Instance instance : sourceInstances) {
|
||||||
|
if (needSync(instance.getMetadata())) {
|
||||||
|
String instanceKey = composeInstanceKey(instance.getIp(), instance.getPort());
|
||||||
|
if (CollectionUtils.isEmpty(instanceKeys) || !instanceKeys.contains(instanceKey)) {
|
||||||
|
register(taskDO, instance);
|
||||||
|
}
|
||||||
|
latestSyncInstance.add(instanceKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (CollectionUtils.isNotEmpty(latestSyncInstance)) {
|
||||||
|
log.info("任务Id:{},已同步实例个数:{}", taskId, latestSyncInstance.size());
|
||||||
|
sourceInstanceSnapshot.put(taskId, latestSyncInstance);
|
||||||
|
} else {
|
||||||
|
// latestSyncInstance为空表示源集群中需要同步的所有实例(即非nacos-sync同步过来的实例)已经下线,清除本地持有快照
|
||||||
|
sourceInstanceSnapshot.remove(taskId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
private void removeInvalidInstance(TaskDO taskDO, List<Instance> sourceInstances) throws Exception {
|
||||||
|
String taskId = taskDO.getTaskId();
|
||||||
|
if (this.sourceInstanceSnapshot.containsKey(taskId)) {
|
||||||
|
Set<String> oldInstanceKeys = this.sourceInstanceSnapshot.get(taskId);
|
||||||
|
Set<String> newInstanceKeys = sourceInstances.stream()
|
||||||
|
.map(instance -> composeInstanceKey(instance.getIp(), instance.getPort()))
|
||||||
|
.collect(Collectors.toSet());
|
||||||
|
oldInstanceKeys.removeAll(newInstanceKeys);
|
||||||
|
if (CollectionUtils.isNotEmpty(oldInstanceKeys)) {
|
||||||
|
log.info("任务Id:{},移除无效同步实例:{}", taskId, oldInstanceKeys);
|
||||||
|
removeInvalidInstance(taskDO, oldInstanceKeys);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public boolean needDelete(Map<String, String> destMetaData, TaskDO taskDO) {
|
||||||
|
return SyncService.super.needDelete(destMetaData, taskDO);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public boolean needSync(Map<String, String> sourceMetaData) {
|
||||||
|
return SyncService.super.needSync(sourceMetaData);
|
||||||
|
}
|
||||||
|
|
||||||
|
public abstract String composeInstanceKey(String ip, int port);
|
||||||
|
|
||||||
|
public abstract void register(TaskDO taskDO, Instance instance);
|
||||||
|
|
||||||
|
public abstract void deregisterInstance(TaskDO taskDO) throws Exception;
|
||||||
|
|
||||||
|
public abstract void removeInvalidInstance(TaskDO taskDO, Set<String> invalidInstanceKeys) throws Exception;
|
||||||
|
|
||||||
|
}
|
|
@ -27,14 +27,18 @@ import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.util.ConsulUtils;
|
import com.alibaba.nacossync.util.ConsulUtils;
|
||||||
|
import com.alibaba.nacossync.util.NacosUtils;
|
||||||
import com.ecwid.consul.v1.ConsulClient;
|
import com.ecwid.consul.v1.ConsulClient;
|
||||||
import com.ecwid.consul.v1.QueryParams;
|
import com.ecwid.consul.v1.QueryParams;
|
||||||
import com.ecwid.consul.v1.Response;
|
import com.ecwid.consul.v1.Response;
|
||||||
import com.ecwid.consul.v1.health.model.HealthService;
|
import com.ecwid.consul.v1.health.model.HealthService;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
|
|
||||||
import java.util.*;
|
import java.util.HashMap;
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Set;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Consul 同步 Nacos
|
* Consul 同步 Nacos
|
||||||
|
@ -46,8 +50,7 @@ import java.util.*;
|
||||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.CONSUL, destinationCluster = ClusterTypeEnum.NACOS)
|
@NacosSyncService(sourceCluster = ClusterTypeEnum.CONSUL, destinationCluster = ClusterTypeEnum.NACOS)
|
||||||
public class ConsulSyncToNacosServiceImpl implements SyncService {
|
public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||||
|
|
||||||
@Autowired
|
private final MetricsManager metricsManager;
|
||||||
private MetricsManager metricsManager;
|
|
||||||
|
|
||||||
private final ConsulServerHolder consulServerHolder;
|
private final ConsulServerHolder consulServerHolder;
|
||||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
|
@ -56,14 +59,15 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||||
|
|
||||||
private final SpecialSyncEventBus specialSyncEventBus;
|
private final SpecialSyncEventBus specialSyncEventBus;
|
||||||
|
|
||||||
@Autowired
|
|
||||||
public ConsulSyncToNacosServiceImpl(ConsulServerHolder consulServerHolder,
|
public ConsulSyncToNacosServiceImpl(ConsulServerHolder consulServerHolder,
|
||||||
SkyWalkerCacheServices skyWalkerCacheServices, NacosServerHolder nacosServerHolder,
|
SkyWalkerCacheServices skyWalkerCacheServices, NacosServerHolder nacosServerHolder,
|
||||||
SpecialSyncEventBus specialSyncEventBus) {
|
SpecialSyncEventBus specialSyncEventBus, MetricsManager metricsManager) {
|
||||||
this.consulServerHolder = consulServerHolder;
|
this.consulServerHolder = consulServerHolder;
|
||||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
this.nacosServerHolder = nacosServerHolder;
|
this.nacosServerHolder = nacosServerHolder;
|
||||||
this.specialSyncEventBus = specialSyncEventBus;
|
this.specialSyncEventBus = specialSyncEventBus;
|
||||||
|
this.metricsManager = metricsManager;
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
|
@ -71,12 +75,15 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||||
|
|
||||||
try {
|
try {
|
||||||
specialSyncEventBus.unsubscribe(taskDO);
|
specialSyncEventBus.unsubscribe(taskDO);
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
|
||||||
List<Instance> allInstances = destNamingService.getAllInstances(taskDO.getServiceName());
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
List<Instance> allInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()));
|
||||||
for (Instance instance : allInstances) {
|
for (Instance instance : allInstances) {
|
||||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||||
|
|
||||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -89,17 +96,17 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean sync(TaskDO taskDO) {
|
public boolean sync(TaskDO taskDO, Integer index) {
|
||||||
try {
|
try {
|
||||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getSourceClusterId(), null);
|
ConsulClient consulClient = consulServerHolder.get(taskDO.getSourceClusterId());
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
Response<List<HealthService>> response =
|
Response<List<HealthService>> response =
|
||||||
consulClient.getHealthServices(taskDO.getServiceName(), true, QueryParams.DEFAULT);
|
consulClient.getHealthServices(taskDO.getServiceName(), true, QueryParams.DEFAULT);
|
||||||
List<HealthService> healthServiceList = response.getValue();
|
List<HealthService> healthServiceList = response.getValue();
|
||||||
Set<String> instanceKeys = new HashSet<>();
|
Set<String> instanceKeys = new HashSet<>();
|
||||||
overrideAllInstance(taskDO, destNamingService, healthServiceList, instanceKeys);
|
overrideAllInstance(taskDO, destNamingService, healthServiceList, instanceKeys);
|
||||||
cleanAllOldInstance(taskDO, destNamingService, instanceKeys);
|
cleanAllOldInstance(taskDO, destNamingService, instanceKeys);
|
||||||
specialSyncEventBus.subscribe(taskDO, this::sync);
|
specialSyncEventBus.subscribe(taskDO, t->sync(t, index));
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("Sync task from consul to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
log.error("Sync task from consul to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
@ -115,7 +122,8 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||||
if (needDelete(instance.getMetadata(), taskDO)
|
if (needDelete(instance.getMetadata(), taskDO)
|
||||||
&& !instanceKeys.contains(composeInstanceKey(instance.getIp(), instance.getPort()))) {
|
&& !instanceKeys.contains(composeInstanceKey(instance.getIp(), instance.getPort()))) {
|
||||||
|
|
||||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -125,6 +133,7 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||||
for (HealthService healthService : healthServiceList) {
|
for (HealthService healthService : healthServiceList) {
|
||||||
if (needSync(ConsulUtils.transferMetadata(healthService.getService().getTags()))) {
|
if (needSync(ConsulUtils.transferMetadata(healthService.getService().getTags()))) {
|
||||||
destNamingService.registerInstance(taskDO.getServiceName(),
|
destNamingService.registerInstance(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
buildSyncInstance(healthService, taskDO));
|
buildSyncInstance(healthService, taskDO));
|
||||||
instanceKeys.add(composeInstanceKey(healthService.getService().getAddress(),
|
instanceKeys.add(composeInstanceKey(healthService.getService().getAddress(),
|
||||||
healthService.getService().getPort()));
|
healthService.getService().getPort()));
|
||||||
|
@ -137,10 +146,10 @@ public class ConsulSyncToNacosServiceImpl implements SyncService {
|
||||||
temp.setIp(instance.getService().getAddress());
|
temp.setIp(instance.getService().getAddress());
|
||||||
temp.setPort(instance.getService().getPort());
|
temp.setPort(instance.getService().getPort());
|
||||||
Map<String, String> metaData = new HashMap<>(ConsulUtils.transferMetadata(instance.getService().getTags()));
|
Map<String, String> metaData = new HashMap<>(ConsulUtils.transferMetadata(instance.getService().getTags()));
|
||||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||||
temp.setMetadata(metaData);
|
temp.setMetadata(metaData);
|
||||||
return temp;
|
return temp;
|
||||||
}
|
}
|
||||||
|
|
|
@ -27,6 +27,7 @@ import com.alibaba.nacossync.extension.holder.EurekaServerHolder;
|
||||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import com.alibaba.nacossync.util.NacosUtils;
|
||||||
import com.netflix.appinfo.InstanceInfo;
|
import com.netflix.appinfo.InstanceInfo;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
@ -71,8 +72,10 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
||||||
|
|
||||||
try {
|
try {
|
||||||
specialSyncEventBus.unsubscribe(taskDO);
|
specialSyncEventBus.unsubscribe(taskDO);
|
||||||
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId(), null);
|
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId());
|
||||||
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
|
||||||
List<InstanceInfo> eurekaInstances = eurekaNamingService.getApplications(taskDO.getServiceName());
|
List<InstanceInfo> eurekaInstances = eurekaNamingService.getApplications(taskDO.getServiceName());
|
||||||
deleteAllInstanceFromEureka(taskDO, destNamingService, eurekaInstances);
|
deleteAllInstanceFromEureka(taskDO, destNamingService, eurekaInstances);
|
||||||
|
|
||||||
|
@ -85,12 +88,15 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean sync(TaskDO taskDO) {
|
public boolean sync(TaskDO taskDO,Integer index) {
|
||||||
try {
|
try {
|
||||||
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId(), null);
|
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
EurekaNamingService eurekaNamingService = eurekaServerHolder.get(taskDO.getSourceClusterId());
|
||||||
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
|
||||||
List<InstanceInfo> eurekaInstances = eurekaNamingService.getApplications(taskDO.getServiceName());
|
List<InstanceInfo> eurekaInstances = eurekaNamingService.getApplications(taskDO.getServiceName());
|
||||||
List<Instance> nacosInstances = destNamingService.getAllInstances(taskDO.getServiceName());
|
List<Instance> nacosInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()));
|
||||||
|
|
||||||
if (CollectionUtils.isEmpty(eurekaInstances)) {
|
if (CollectionUtils.isEmpty(eurekaInstances)) {
|
||||||
// Clear all instance from Nacos
|
// Clear all instance from Nacos
|
||||||
|
@ -102,7 +108,7 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
addValidInstance(taskDO, destNamingService, eurekaInstances);
|
addValidInstance(taskDO, destNamingService, eurekaInstances);
|
||||||
}
|
}
|
||||||
specialSyncEventBus.subscribe(taskDO, this::sync);
|
specialSyncEventBus.subscribe(taskDO, t->sync(t, index));
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("sync task from eureka to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
log.error("sync task from eureka to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
@ -117,18 +123,25 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
||||||
if (needSync(instance.getMetadata())) {
|
if (needSync(instance.getMetadata())) {
|
||||||
log.info("Add service instance from Eureka, serviceName={}, Ip={}, port={}",
|
log.info("Add service instance from Eureka, serviceName={}, Ip={}, port={}",
|
||||||
instance.getAppName(), instance.getIPAddr(), instance.getPort());
|
instance.getAppName(), instance.getIPAddr(), instance.getPort());
|
||||||
destNamingService.registerInstance(taskDO.getServiceName(), buildSyncInstance(instance, taskDO));
|
destNamingService.registerInstance(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), buildSyncInstance(instance,
|
||||||
|
taskDO));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private void deleteAllInstanceFromEureka(TaskDO taskDO, NamingService destNamingService, List<InstanceInfo> eurekaInstances)
|
private void deleteAllInstanceFromEureka(TaskDO taskDO, NamingService destNamingService,
|
||||||
throws NacosException {
|
List<InstanceInfo> eurekaInstances)
|
||||||
|
throws NacosException {
|
||||||
|
if (CollectionUtils.isEmpty(eurekaInstances)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
for (InstanceInfo instance : eurekaInstances) {
|
for (InstanceInfo instance : eurekaInstances) {
|
||||||
if (needSync(instance.getMetadata())) {
|
if (needSync(instance.getMetadata())) {
|
||||||
log.info("Delete service instance from Eureka, serviceName={}, Ip={}, port={}",
|
log.info("Delete service instance from Eureka, serviceName={}, Ip={}, port={}",
|
||||||
instance.getAppName(), instance.getIPAddr(), instance.getPort());
|
instance.getAppName(), instance.getIPAddr(), instance.getPort());
|
||||||
destNamingService.deregisterInstance(taskDO.getServiceName(), buildSyncInstance(instance, taskDO));
|
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), buildSyncInstance(instance, taskDO));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -139,7 +152,8 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
||||||
if (!isExistInEurekaInstance(eurekaInstances, instance) && needDelete(instance.getMetadata(), taskDO)) {
|
if (!isExistInEurekaInstance(eurekaInstances, instance) && needDelete(instance.getMetadata(), taskDO)) {
|
||||||
log.info("Remove invalid service instance from Nacos, serviceName={}, Ip={}, port={}",
|
log.info("Remove invalid service instance from Nacos, serviceName={}, Ip={}, port={}",
|
||||||
instance.getServiceName(), instance.getIp(), instance.getPort());
|
instance.getServiceName(), instance.getIp(), instance.getPort());
|
||||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -154,7 +168,8 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
||||||
throws NacosException {
|
throws NacosException {
|
||||||
for (Instance instance : allInstances) {
|
for (Instance instance : allInstances) {
|
||||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance);
|
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||||
|
NacosUtils.getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
@ -168,10 +183,10 @@ public class EurekaSyncToNacosServiceImpl implements SyncService {
|
||||||
temp.setHealthy(true);
|
temp.setHealthy(true);
|
||||||
|
|
||||||
Map<String, String> metaData = new HashMap<>(instance.getMetadata());
|
Map<String, String> metaData = new HashMap<>(instance.getMetadata());
|
||||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||||
temp.setMetadata(metaData);
|
temp.setMetadata(metaData);
|
||||||
return temp;
|
return temp;
|
||||||
}
|
}
|
||||||
|
|
|
@ -10,21 +10,15 @@
|
||||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
* specific language governing permissions and limitations under the License.
|
* specific language governing permissions and limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.extension.impl;
|
package com.alibaba.nacossync.extension.impl;
|
||||||
|
|
||||||
import com.alibaba.nacos.api.naming.NamingService;
|
|
||||||
import com.alibaba.nacos.api.naming.listener.EventListener;
|
|
||||||
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
|
||||||
import com.alibaba.nacos.api.naming.pojo.Instance;
|
import com.alibaba.nacos.api.naming.pojo.Instance;
|
||||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
|
||||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||||
import com.alibaba.nacossync.extension.SyncService;
|
|
||||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||||
import com.alibaba.nacossync.extension.holder.ConsulServerHolder;
|
import com.alibaba.nacossync.extension.holder.ConsulServerHolder;
|
||||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.util.ConsulUtils;
|
import com.alibaba.nacossync.util.ConsulUtils;
|
||||||
import com.ecwid.consul.v1.ConsulClient;
|
import com.ecwid.consul.v1.ConsulClient;
|
||||||
|
@ -33,15 +27,13 @@ import com.ecwid.consul.v1.Response;
|
||||||
import com.ecwid.consul.v1.agent.model.NewService;
|
import com.ecwid.consul.v1.agent.model.NewService;
|
||||||
import com.ecwid.consul.v1.health.model.HealthService;
|
import com.ecwid.consul.v1.health.model.HealthService;
|
||||||
import com.google.common.collect.Lists;
|
import com.google.common.collect.Lists;
|
||||||
import java.net.URLEncoder;
|
|
||||||
import java.nio.charset.StandardCharsets;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
|
||||||
import java.util.HashSet;
|
import java.io.UnsupportedEncodingException;
|
||||||
|
import java.net.URLEncoder;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Map;
|
|
||||||
import java.util.Set;
|
import java.util.Set;
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
|
||||||
import java.util.stream.Collectors;
|
import java.util.stream.Collectors;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -49,105 +41,64 @@ import java.util.stream.Collectors;
|
||||||
*/
|
*/
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.CONSUL)
|
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.CONSUL)
|
||||||
public class NacosSyncToConsulServiceImpl implements SyncService {
|
public class NacosSyncToConsulServiceImpl extends AbstractNacosSync {
|
||||||
private Map<String, EventListener> nacosListenerMap = new ConcurrentHashMap<>();
|
|
||||||
|
|
||||||
private final MetricsManager metricsManager;
|
|
||||||
|
private static final String DELIMITER = "=";
|
||||||
|
|
||||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
|
|
||||||
private final NacosServerHolder nacosServerHolder;
|
|
||||||
private final ConsulServerHolder consulServerHolder;
|
private final ConsulServerHolder consulServerHolder;
|
||||||
|
|
||||||
public NacosSyncToConsulServiceImpl(MetricsManager metricsManager, SkyWalkerCacheServices skyWalkerCacheServices,
|
public NacosSyncToConsulServiceImpl(SkyWalkerCacheServices skyWalkerCacheServices,
|
||||||
NacosServerHolder nacosServerHolder, ConsulServerHolder consulServerHolder) {
|
ConsulServerHolder consulServerHolder) {
|
||||||
this.metricsManager = metricsManager;
|
|
||||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
this.nacosServerHolder = nacosServerHolder;
|
|
||||||
this.consulServerHolder = consulServerHolder;
|
this.consulServerHolder = consulServerHolder;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean delete(TaskDO taskDO) {
|
public String composeInstanceKey(String ip, int port) {
|
||||||
try {
|
return String.join(":", ip, String.valueOf(port));
|
||||||
|
}
|
||||||
|
|
||||||
NamingService sourceNamingService =
|
@Override
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
public void register(TaskDO taskDO, Instance instance) {
|
||||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
consulClient.agentServiceRegister(buildSyncInstance(instance, taskDO));
|
||||||
|
}
|
||||||
|
|
||||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
@Override
|
||||||
|
public void deregisterInstance(TaskDO taskDO) throws Exception {
|
||||||
|
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
Response<List<HealthService>> serviceResponse = consulClient.getHealthServices(taskDO.getServiceName(), true,
|
||||||
|
QueryParams.DEFAULT);
|
||||||
|
List<HealthService> healthServices = serviceResponse.getValue();
|
||||||
|
for (HealthService healthService : healthServices) {
|
||||||
|
|
||||||
// 删除目标集群中同步的实例列表
|
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)) {
|
||||||
Response<List<HealthService>> serviceResponse =
|
consulClient.agentServiceDeregister(
|
||||||
consulClient.getHealthServices(taskDO.getServiceName(), true, QueryParams.DEFAULT);
|
URLEncoder.encode(healthService.getService().getId(), StandardCharsets.UTF_8));
|
||||||
List<HealthService> healthServices = serviceResponse.getValue();
|
|
||||||
for (HealthService healthService : healthServices) {
|
|
||||||
|
|
||||||
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)) {
|
|
||||||
consulClient.agentServiceDeregister(URLEncoder
|
|
||||||
.encode(healthService.getService().getId(), StandardCharsets.UTF_8.name()));
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("delete a task from nacos to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
|
||||||
return false;
|
|
||||||
}
|
}
|
||||||
return true;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean sync(TaskDO taskDO) {
|
public void removeInvalidInstance(TaskDO taskDO, Set<String> invalidInstanceKeys)
|
||||||
try {
|
throws UnsupportedEncodingException {
|
||||||
NamingService sourceNamingService =
|
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId());
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
Response<List<HealthService>> serviceResponse = consulClient.getHealthServices(taskDO.getServiceName(), true,
|
||||||
ConsulClient consulClient = consulServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
QueryParams.DEFAULT);
|
||||||
|
List<HealthService> healthServices = serviceResponse.getValue();
|
||||||
|
for (HealthService healthService : healthServices) {
|
||||||
|
|
||||||
nacosListenerMap.putIfAbsent(taskDO.getTaskId(), event -> {
|
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)
|
||||||
if (event instanceof NamingEvent) {
|
&& !invalidInstanceKeys.contains(composeInstanceKey(healthService.getService().getAddress(),
|
||||||
try {
|
healthService.getService().getPort()))) {
|
||||||
Set<String> instanceKeySet = new HashSet();
|
consulClient.agentServiceDeregister(
|
||||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName());
|
URLEncoder.encode(healthService.getService().getId(), StandardCharsets.UTF_8));
|
||||||
// 先将新的注册一遍
|
}
|
||||||
for (Instance instance : sourceInstances) {
|
|
||||||
if (needSync(instance.getMetadata())) {
|
|
||||||
consulClient.agentServiceRegister(buildSyncInstance(instance, taskDO));
|
|
||||||
instance.getInstanceId();
|
|
||||||
instanceKeySet.add(composeInstanceKey(instance.getIp(), instance.getPort()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// 再将不存在的删掉
|
|
||||||
Response<List<HealthService>> serviceResponse =
|
|
||||||
consulClient.getHealthServices(taskDO.getServiceName(), true, QueryParams.DEFAULT);
|
|
||||||
List<HealthService> healthServices = serviceResponse.getValue();
|
|
||||||
for (HealthService healthService : healthServices) {
|
|
||||||
|
|
||||||
if (needDelete(ConsulUtils.transferMetadata(healthService.getService().getTags()), taskDO)
|
|
||||||
&& !instanceKeySet.contains(composeInstanceKey(healthService.getService().getAddress(),
|
|
||||||
healthService.getService().getPort()))) {
|
|
||||||
consulClient.agentServiceDeregister(URLEncoder
|
|
||||||
.encode(healthService.getService().getId(), StandardCharsets.UTF_8.toString()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("event process fail, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
sourceNamingService.subscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("sync task from nacos to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
|
||||||
return false;
|
|
||||||
}
|
}
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
private String composeInstanceKey(String ip, int port) {
|
|
||||||
return ip + ":" + port;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public NewService buildSyncInstance(Instance instance, TaskDO taskDO) {
|
public NewService buildSyncInstance(Instance instance, TaskDO taskDO) {
|
||||||
|
@ -158,11 +109,11 @@ public class NacosSyncToConsulServiceImpl implements SyncService {
|
||||||
newService.setId(instance.getInstanceId());
|
newService.setId(instance.getInstanceId());
|
||||||
List<String> tags = Lists.newArrayList();
|
List<String> tags = Lists.newArrayList();
|
||||||
tags.addAll(instance.getMetadata().entrySet().stream()
|
tags.addAll(instance.getMetadata().entrySet().stream()
|
||||||
.map(entry -> String.join("=", entry.getKey(), entry.getValue())).collect(Collectors.toList()));
|
.map(entry -> String.join(DELIMITER, entry.getKey(), entry.getValue())).collect(Collectors.toList()));
|
||||||
tags.add(String.join("=", SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId()));
|
tags.add(String.join(DELIMITER, SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId()));
|
||||||
tags.add(String.join("=", SkyWalkerConstants.SYNC_SOURCE_KEY,
|
tags.add(String.join(DELIMITER, SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode()));
|
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode()));
|
||||||
tags.add(String.join("=", SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId()));
|
tags.add(String.join(DELIMITER, SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId()));
|
||||||
newService.setTags(tags);
|
newService.setTags(tags);
|
||||||
return newService;
|
return newService;
|
||||||
}
|
}
|
||||||
|
|
|
@ -10,23 +10,17 @@
|
||||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
* specific language governing permissions and limitations under the License.
|
* specific language governing permissions and limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.extension.impl;
|
package com.alibaba.nacossync.extension.impl;
|
||||||
|
|
||||||
import com.alibaba.nacos.api.naming.NamingService;
|
|
||||||
import com.alibaba.nacos.api.naming.listener.Event;
|
|
||||||
import com.alibaba.nacos.api.naming.listener.EventListener;
|
|
||||||
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
|
||||||
import com.alibaba.nacos.api.naming.pojo.Instance;
|
import com.alibaba.nacos.api.naming.pojo.Instance;
|
||||||
|
import com.alibaba.nacos.common.utils.CollectionUtils;
|
||||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
|
||||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||||
import com.alibaba.nacossync.extension.SyncService;
|
|
||||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||||
import com.alibaba.nacossync.extension.eureka.EurekaNamingService;
|
import com.alibaba.nacossync.extension.eureka.EurekaNamingService;
|
||||||
import com.alibaba.nacossync.extension.holder.EurekaServerHolder;
|
import com.alibaba.nacossync.extension.holder.EurekaServerHolder;
|
||||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.netflix.appinfo.DataCenterInfo;
|
import com.netflix.appinfo.DataCenterInfo;
|
||||||
import com.netflix.appinfo.InstanceInfo;
|
import com.netflix.appinfo.InstanceInfo;
|
||||||
|
@ -34,169 +28,94 @@ import com.netflix.appinfo.LeaseInfo;
|
||||||
import com.netflix.appinfo.MyDataCenterInfo;
|
import com.netflix.appinfo.MyDataCenterInfo;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
|
||||||
import java.util.*;
|
import java.util.HashMap;
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Set;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author zhanglong
|
* @author zhanglong
|
||||||
*/
|
*/
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.EUREKA)
|
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.EUREKA)
|
||||||
public class NacosSyncToEurekaServiceImpl implements SyncService {
|
public class NacosSyncToEurekaServiceImpl extends AbstractNacosSync {
|
||||||
private final Map<String, EventListener> nacosListenerMap = new ConcurrentHashMap<>();
|
|
||||||
|
|
||||||
private final MetricsManager metricsManager;
|
|
||||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
|
||||||
private final NacosServerHolder nacosServerHolder;
|
|
||||||
private final EurekaServerHolder eurekaServerHolder;
|
private final EurekaServerHolder eurekaServerHolder;
|
||||||
|
|
||||||
public NacosSyncToEurekaServiceImpl(MetricsManager metricsManager, SkyWalkerCacheServices skyWalkerCacheServices,
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
NacosServerHolder nacosServerHolder, EurekaServerHolder eurekaServerHolder) {
|
|
||||||
this.metricsManager = metricsManager;
|
public NacosSyncToEurekaServiceImpl(EurekaServerHolder eurekaServerHolder,
|
||||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
SkyWalkerCacheServices skyWalkerCacheServices) {
|
||||||
this.nacosServerHolder = nacosServerHolder;
|
|
||||||
this.eurekaServerHolder = eurekaServerHolder;
|
this.eurekaServerHolder = eurekaServerHolder;
|
||||||
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean delete(TaskDO taskDO) {
|
public String composeInstanceKey(String ip, int port) {
|
||||||
try {
|
|
||||||
NamingService sourceNamingService =
|
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
|
||||||
EurekaNamingService destNamingService =
|
|
||||||
eurekaServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
|
||||||
|
|
||||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
|
||||||
// 删除目标集群中同步的实例列表
|
|
||||||
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
|
||||||
if (allInstances != null) {
|
|
||||||
for (InstanceInfo instance : allInstances) {
|
|
||||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
|
||||||
destNamingService.deregisterInstance(instance);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("delete task from nacos to eureka was failed, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public boolean sync(TaskDO taskDO) {
|
|
||||||
try {
|
|
||||||
NamingService sourceNamingService =
|
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
|
||||||
EurekaNamingService destNamingService =
|
|
||||||
eurekaServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
|
||||||
|
|
||||||
nacosListenerMap.putIfAbsent(taskDO.getTaskId(), event -> {
|
|
||||||
processNamingEvent(taskDO, sourceNamingService, destNamingService, event);
|
|
||||||
});
|
|
||||||
|
|
||||||
sourceNamingService.subscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("sync task from eureka to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
private void processNamingEvent(TaskDO taskDO, NamingService sourceNamingService,
|
|
||||||
EurekaNamingService destNamingService, Event event) {
|
|
||||||
if (event instanceof NamingEvent) {
|
|
||||||
try {
|
|
||||||
Set<String> instanceKeySet = new HashSet<>();
|
|
||||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName());
|
|
||||||
// 先将新的注册一遍
|
|
||||||
addAllNewInstance(taskDO, destNamingService, instanceKeySet, sourceInstances);
|
|
||||||
// 再将不存在的删掉
|
|
||||||
ifNecessaryDelete(taskDO, destNamingService, instanceKeySet);
|
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("event process fail, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private void ifNecessaryDelete(TaskDO taskDO, EurekaNamingService destNamingService, Set<String> instanceKeySet) {
|
|
||||||
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
|
||||||
if (allInstances != null){
|
|
||||||
for (InstanceInfo instance : allInstances) {
|
|
||||||
if (needDelete(instance.getMetadata(), taskDO) && !instanceKeySet.contains(composeInstanceKey(instance.getIPAddr(),
|
|
||||||
instance.getPort()))) {
|
|
||||||
destNamingService.deregisterInstance(instance);
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private void addAllNewInstance(TaskDO taskDO, EurekaNamingService destNamingService, Set<String> instanceKeySet,
|
|
||||||
List<Instance> sourceInstances) {
|
|
||||||
for (Instance instance : sourceInstances) {
|
|
||||||
if (needSync(instance.getMetadata())) {
|
|
||||||
destNamingService.registerInstance(buildSyncInstance(instance, taskDO));
|
|
||||||
instanceKeySet.add(composeInstanceKey(instance.getIp(), instance.getPort()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private String composeInstanceKey(String ip, int port) {
|
|
||||||
return ip + ":" + port;
|
return ip + ":" + port;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void register(TaskDO taskDO, Instance instance) {
|
||||||
|
EurekaNamingService destNamingService = eurekaServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
destNamingService.registerInstance(buildSyncInstance(instance, taskDO));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void deregisterInstance(TaskDO taskDO) {
|
||||||
|
EurekaNamingService destNamingService = eurekaServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
||||||
|
if (allInstances != null) {
|
||||||
|
for (InstanceInfo instance : allInstances) {
|
||||||
|
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||||
|
destNamingService.deregisterInstance(instance);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void removeInvalidInstance(TaskDO taskDO, Set<String> invalidInstanceKeys) {
|
||||||
|
EurekaNamingService destNamingService = eurekaServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
List<InstanceInfo> allInstances = destNamingService.getApplications(taskDO.getServiceName());
|
||||||
|
if (CollectionUtils.isNotEmpty(allInstances)) {
|
||||||
|
for (InstanceInfo instance : allInstances) {
|
||||||
|
if (needDelete(instance.getMetadata(), taskDO) && invalidInstanceKeys.contains(
|
||||||
|
composeInstanceKey(instance.getIPAddr(), instance.getPort()))) {
|
||||||
|
destNamingService.deregisterInstance(instance);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
private InstanceInfo buildSyncInstance(Instance instance, TaskDO taskDO) {
|
private InstanceInfo buildSyncInstance(Instance instance, TaskDO taskDO) {
|
||||||
DataCenterInfo dataCenterInfo = new MyDataCenterInfo(DataCenterInfo.Name.MyOwn);
|
DataCenterInfo dataCenterInfo = new MyDataCenterInfo(DataCenterInfo.Name.MyOwn);
|
||||||
final Map<String, String> instanceMetadata = instance.getMetadata();
|
final Map<String, String> instanceMetadata = instance.getMetadata();
|
||||||
HashMap<String, String> metadata = new HashMap<>(16);
|
HashMap<String, String> metadata = new HashMap<>(16);
|
||||||
metadata.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
metadata.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||||
metadata.put(SkyWalkerConstants.SYNC_SOURCE_KEY, skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
metadata.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||||
metadata.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||||
|
metadata.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||||
metadata.putAll(instanceMetadata);
|
metadata.putAll(instanceMetadata);
|
||||||
String homePageUrl = obtainHomePageUrl(instance, instanceMetadata);
|
String homePageUrl = obtainHomePageUrl(instance, instanceMetadata);
|
||||||
String serviceName = taskDO.getServiceName();
|
String serviceName = taskDO.getServiceName();
|
||||||
|
|
||||||
return new InstanceInfo(
|
return new InstanceInfo(instance.getIp() + ":" + serviceName + ":" + instance.getPort(), serviceName, null,
|
||||||
instance.getIp() + ":" + serviceName + ":" + instance.getPort(),
|
instance.getIp(), null, new InstanceInfo.PortWrapper(true, instance.getPort()), null,
|
||||||
serviceName,
|
homePageUrl + "/actuator/env", homePageUrl + "/actuator/info", homePageUrl + "/actuator/health", null,
|
||||||
null,
|
serviceName, serviceName, 1, dataCenterInfo, instance.getIp(), InstanceInfo.InstanceStatus.UP,
|
||||||
instance.getIp(),
|
InstanceInfo.InstanceStatus.UNKNOWN, null, new LeaseInfo(30, 90, 0L, 0L, 0L, 0L, 0L), false, metadata,
|
||||||
null,
|
System.currentTimeMillis(), System.currentTimeMillis(), null, null);
|
||||||
new InstanceInfo.PortWrapper(true, instance.getPort()),
|
|
||||||
null,
|
|
||||||
homePageUrl+"/actuator/env",
|
|
||||||
homePageUrl + "/actuator/info",
|
|
||||||
homePageUrl + "/actuator/health",
|
|
||||||
null,
|
|
||||||
serviceName,
|
|
||||||
serviceName,
|
|
||||||
1,
|
|
||||||
dataCenterInfo,
|
|
||||||
instance.getIp(),
|
|
||||||
InstanceInfo.InstanceStatus.UP,
|
|
||||||
InstanceInfo.InstanceStatus.UNKNOWN,
|
|
||||||
null,
|
|
||||||
new LeaseInfo(30, 90,
|
|
||||||
0L, 0L, 0L, 0L, 0L),
|
|
||||||
false,
|
|
||||||
metadata,
|
|
||||||
System.currentTimeMillis(),
|
|
||||||
System.currentTimeMillis(),
|
|
||||||
null,
|
|
||||||
null
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private String obtainHomePageUrl(Instance instance, Map<String, String> instanceMetadata) {
|
private String obtainHomePageUrl(Instance instance, Map<String, String> instanceMetadata) {
|
||||||
final String managementContextPath =
|
final String managementContextPath = obtainManagementContextPath(instanceMetadata);
|
||||||
obtainManagementContextPath(instanceMetadata);
|
|
||||||
final String managementPort = instanceMetadata.getOrDefault(SkyWalkerConstants.MANAGEMENT_PORT_KEY,
|
final String managementPort = instanceMetadata.getOrDefault(SkyWalkerConstants.MANAGEMENT_PORT_KEY,
|
||||||
String.valueOf(instance.getPort()));
|
String.valueOf(instance.getPort()));
|
||||||
return String.format("http://%s:%s%s",instance.getIp(),managementPort,managementContextPath);
|
return String.format("http://%s:%s%s", instance.getIp(), managementPort, managementContextPath);
|
||||||
}
|
}
|
||||||
|
|
||||||
private String obtainManagementContextPath(Map<String, String> instanceMetadata) {
|
private String obtainManagementContextPath(Map<String, String> instanceMetadata) {
|
||||||
|
@ -208,5 +127,4 @@ public class NacosSyncToEurekaServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -10,6 +10,7 @@
|
||||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
* specific language governing permissions and limitations under the License.
|
* specific language governing permissions and limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.extension.impl;
|
package com.alibaba.nacossync.extension.impl;
|
||||||
|
|
||||||
import com.alibaba.nacos.api.exception.NacosException;
|
import com.alibaba.nacos.api.exception.NacosException;
|
||||||
|
@ -22,23 +23,32 @@ import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||||
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||||
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
import com.alibaba.nacossync.extension.SyncService;
|
import com.alibaba.nacossync.extension.SyncService;
|
||||||
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
import com.alibaba.nacossync.extension.annotation.NacosSyncService;
|
||||||
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.util.Collections;
|
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||||
|
import com.alibaba.nacossync.util.StringUtils;
|
||||||
|
import com.google.common.base.Stopwatch;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.beans.factory.DisposableBean;
|
||||||
|
import org.springframework.beans.factory.InitializingBean;
|
||||||
|
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
import java.util.Collection;
|
import java.util.Collection;
|
||||||
import java.util.HashMap;
|
import java.util.HashMap;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Map;
|
import java.util.Map;
|
||||||
import java.util.Set;
|
|
||||||
import java.util.TreeSet;
|
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
import java.util.concurrent.Executors;
|
||||||
|
import java.util.concurrent.ScheduledExecutorService;
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
import java.util.stream.Collectors;
|
import java.util.stream.Collectors;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
import static com.alibaba.nacossync.constant.SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author yangyshdan
|
* @author yangyshdan
|
||||||
|
@ -47,42 +57,135 @@ import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.NACOS)
|
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.NACOS)
|
||||||
public class NacosSyncToNacosServiceImpl implements SyncService {
|
public class NacosSyncToNacosServiceImpl implements SyncService, InitializingBean, DisposableBean {
|
||||||
|
|
||||||
private Map<String, EventListener> listenerMap = new ConcurrentHashMap<>();
|
private final Map<String, EventListener> listenerMap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
private final Map<String, Set<String>> sourceInstanceSnapshot = new ConcurrentHashMap<>();
|
private final Map<String, Integer> syncTaskTap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
@Autowired
|
private final ConcurrentHashMap<String, TaskDO> allSyncTaskMap = new ConcurrentHashMap<>();
|
||||||
private MetricsManager metricsManager;
|
|
||||||
|
|
||||||
@Autowired
|
private ScheduledExecutorService executorService;
|
||||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
|
||||||
|
|
||||||
@Autowired
|
private final MetricsManager metricsManager;
|
||||||
private NacosServerHolder nacosServerHolder;
|
|
||||||
|
private final NacosServerHolder nacosServerHolder;
|
||||||
|
|
||||||
|
private final ClusterAccessService clusterAccessService;
|
||||||
|
|
||||||
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
|
|
||||||
|
public NacosSyncToNacosServiceImpl(MetricsManager metricsManager, NacosServerHolder nacosServerHolder,
|
||||||
|
ClusterAccessService clusterAccessService, SkyWalkerCacheServices skyWalkerCacheServices) {
|
||||||
|
this.metricsManager = metricsManager;
|
||||||
|
this.nacosServerHolder = nacosServerHolder;
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Due to network issues or other reasons, the Nacos Sync synchronization tasks may fail,
|
||||||
|
* resulting in the target cluster's registry missing synchronized instances.
|
||||||
|
* To prevent the target cluster's registry from missing synchronized instances for an extended period,
|
||||||
|
* a fallback worker thread is started every 5 minutes to execute all synchronization tasks.
|
||||||
|
*/
|
||||||
|
@Override
|
||||||
|
public void afterPropertiesSet() {
|
||||||
|
initializeExecutorService();
|
||||||
|
scheduleSyncTasks();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void destroy() {
|
||||||
|
if (executorService != null && !executorService.isShutdown()) {
|
||||||
|
executorService.shutdown();
|
||||||
|
try {
|
||||||
|
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||||
|
executorService.shutdownNow();
|
||||||
|
}
|
||||||
|
} catch (InterruptedException e) {
|
||||||
|
executorService.shutdownNow();
|
||||||
|
Thread.currentThread().interrupt();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void initializeExecutorService() {
|
||||||
|
executorService = Executors.newSingleThreadScheduledExecutor(r -> {
|
||||||
|
Thread t = new Thread(r);
|
||||||
|
t.setDaemon(true);
|
||||||
|
t.setName("com.alibaba.nacossync.basic.synctask");
|
||||||
|
return t;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private void scheduleSyncTasks() {
|
||||||
|
executorService.scheduleWithFixedDelay(this::executeSyncTasks, 0, 300, TimeUnit.SECONDS);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void executeSyncTasks() {
|
||||||
|
if (allSyncTaskMap.isEmpty()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
Collection<TaskDO> taskCollections = allSyncTaskMap.values();
|
||||||
|
List<TaskDO> taskDOList = new ArrayList<>(taskCollections);
|
||||||
|
|
||||||
|
if (CollectionUtils.isNotEmpty(taskDOList)) {
|
||||||
|
BatchTaskExecutor.batchOperation(taskDOList, this::executeTask);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void executeTask(TaskDO task) {
|
||||||
|
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||||
|
String taskId = task.getTaskId();
|
||||||
|
try {
|
||||||
|
NamingService sourceNamingService = nacosServerHolder.get(task.getSourceClusterId());
|
||||||
|
NamingService destNamingService = nacosServerHolder.get(task.getDestClusterId());
|
||||||
|
doSync(taskId, task, sourceNamingService, destNamingService);
|
||||||
|
} catch (NacosException e) {
|
||||||
|
log.error("sync task from nacos to nacos failed, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Unexpected error during sync task, taskId:{}", taskId, e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
} finally {
|
||||||
|
stopwatch.stop();
|
||||||
|
log.debug("Task execution time for taskId {}: {} ms", taskId, stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean delete(TaskDO taskDO) {
|
public boolean delete(TaskDO taskDO) {
|
||||||
try {
|
try {
|
||||||
|
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||||
|
int level = clusterAccessService.findClusterLevel(taskDO.getSourceClusterId());
|
||||||
|
String taskId = taskDO.getTaskId();
|
||||||
|
|
||||||
NamingService sourceNamingService =
|
//Handle individual service
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
if (StringUtils.isEmpty(taskId)) {
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
log.warn("taskId is null data synchronization is not currently performed.{}", taskId);
|
||||||
//移除订阅
|
return false;
|
||||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), listenerMap.remove(taskDO.getTaskId()));
|
}
|
||||||
sourceInstanceSnapshot.remove(taskDO.getTaskId());
|
EventListener listener = listenerMap.remove(taskId);
|
||||||
|
if (listener!= null) {
|
||||||
// 删除目标集群中同步的实例列表
|
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()), listener);
|
||||||
|
}
|
||||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
new ArrayList<>(), false);
|
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), false);
|
||||||
|
|
||||||
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
for (Instance instance : sourceInstances) {
|
for (Instance instance : sourceInstances) {
|
||||||
if (needSync(instance.getMetadata())) {
|
if (needSync(instance.getMetadata(), level, taskDO.getDestClusterId())) {
|
||||||
destNamingService.deregisterInstance(taskDO.getServiceName(), instance.getIp(), instance.getPort());
|
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Remove all tasks that need to be synchronized.
|
||||||
|
allSyncTaskMap.remove(taskId);
|
||||||
|
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("delete task from nacos to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
log.error("delete task from nacos to nacos was failed, operationalId:{}", taskDO.getOperationId(), e);
|
||||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
@ -90,48 +193,37 @@ public class NacosSyncToNacosServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean sync(TaskDO taskDO) {
|
public boolean sync(TaskDO taskDO, Integer index) {
|
||||||
|
log.info("Thread {} started synchronization at {}", Thread.currentThread().getId(), System.currentTimeMillis());
|
||||||
String taskId = taskDO.getTaskId();
|
String taskId = taskDO.getTaskId();
|
||||||
try {
|
try {
|
||||||
NamingService sourceNamingService =
|
NamingService sourceNamingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
allSyncTaskMap.put(taskId, taskDO);
|
||||||
|
// To prevent issues where tasks paused for synchronization, newly created tasks after deletion,
|
||||||
|
// or resynchronization tasks do not receive new events and hence cannot synchronize,
|
||||||
|
// perform a full synchronization of tasks before subscribing to events each time.
|
||||||
|
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||||
|
doSync(taskId, taskDO, sourceNamingService, destNamingService);
|
||||||
|
log.debug("Time taken to synchronize a service registration: {} ms",
|
||||||
|
stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||||
this.listenerMap.putIfAbsent(taskId, event -> {
|
this.listenerMap.putIfAbsent(taskId, event -> {
|
||||||
if (event instanceof NamingEvent) {
|
if (event instanceof NamingEvent) {
|
||||||
|
NamingEvent namingEvent = (NamingEvent) event;
|
||||||
|
log.info("Detected changes in service {} information, taskId: {}, number of instances: {}, initiating synchronization",
|
||||||
|
namingEvent.getServiceName(), taskId,
|
||||||
|
namingEvent.getInstances() == null ? null : namingEvent.getInstances().size());
|
||||||
try {
|
try {
|
||||||
|
doSync(taskId, taskDO, sourceNamingService, destNamingService);
|
||||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
log.info("Detected synchronization end for service {}", namingEvent.getServiceName());
|
||||||
new ArrayList<>(), false);
|
|
||||||
// 先删除不存在的
|
|
||||||
this.removeInvalidInstance(taskDO, taskId, destNamingService, sourceInstances);
|
|
||||||
Set<String> latestSyncInstance = new TreeSet<>();
|
|
||||||
//再次添加新实例
|
|
||||||
Set<String> instanceKeys = sourceInstanceSnapshot.get(taskId);
|
|
||||||
for (Instance instance : sourceInstances) {
|
|
||||||
if (needSync(instance.getMetadata())) {
|
|
||||||
String instanceKey = composeInstanceKey(instance);
|
|
||||||
if (CollectionUtils.isEmpty(instanceKeys) || !instanceKeys.contains(instanceKey)) {
|
|
||||||
destNamingService.registerInstance(taskDO.getServiceName(),
|
|
||||||
buildSyncInstance(instance, taskDO));
|
|
||||||
}
|
|
||||||
latestSyncInstance.add(instanceKey);
|
|
||||||
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (CollectionUtils.isNotEmpty(latestSyncInstance)) {
|
|
||||||
|
|
||||||
log.info("任务Id:{},已同步实例个数:{}", taskId, latestSyncInstance.size());
|
|
||||||
sourceInstanceSnapshot.put(taskId, latestSyncInstance);
|
|
||||||
}
|
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("event process fail, taskId:{}", taskId, e);
|
log.error("event process fail, taskId:{}", taskId, e);
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
sourceNamingService.subscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
sourceNamingService.subscribe(taskDO.getServiceName(), listenerMap.get(taskId));
|
listenerMap.get(taskId));
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("sync task from nacos to nacos was failed, taskId:{}", taskId, e);
|
log.error("sync task from nacos to nacos was failed, taskId:{}", taskId, e);
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
@ -140,30 +232,191 @@ public class NacosSyncToNacosServiceImpl implements SyncService {
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
private void removeInvalidInstance(TaskDO taskDO, String taskId, NamingService destNamingService,
|
|
||||||
List<Instance> sourceInstances) throws NacosException {
|
|
||||||
if (this.sourceInstanceSnapshot.containsKey(taskId)) {
|
|
||||||
Set<String> oldInstanceKeys = this.sourceInstanceSnapshot.get(taskId);
|
|
||||||
List<String> newInstanceKeys = sourceInstances.stream().map(this::composeInstanceKey)
|
|
||||||
.collect(Collectors.toList());
|
|
||||||
Collection<String> instanceKeys = Collections.subtract(oldInstanceKeys, newInstanceKeys);
|
|
||||||
for (String instanceKey : instanceKeys) {
|
|
||||||
log.info("任务Id:{},移除无效同步实例:{}", taskId, instanceKey);
|
|
||||||
String[] split = instanceKey.split(":", -1);
|
|
||||||
destNamingService.deregisterInstance(taskDO.getServiceName(), split[0],
|
|
||||||
Integer.parseInt(split[1]));
|
|
||||||
|
|
||||||
|
private void doSync(String taskId, TaskDO taskDO, NamingService sourceNamingService,
|
||||||
|
NamingService destNamingService) throws NacosException {
|
||||||
|
if (syncTaskTap.putIfAbsent(taskId, 1) != null) {
|
||||||
|
log.info("Task ID:{} - the previous synchronization task has not finished yet", taskId);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
|
||||||
|
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||||
|
|
||||||
|
int level = clusterAccessService.findClusterLevel(taskDO.getSourceClusterId());
|
||||||
|
if (CollectionUtils.isNotEmpty(sourceInstances) && sourceInstances.get(0).isEphemeral()) {
|
||||||
|
// Handle batch data synchronization of ephemeral instances, need to get all current service instances.
|
||||||
|
// TODO: When the Client is version 1.x, execute the same synchronization method as persistent instances.
|
||||||
|
handlerPersistenceInstance(taskDO, destNamingService, sourceInstances, level);
|
||||||
|
} else if (CollectionUtils.isEmpty(sourceInstances)) {
|
||||||
|
// If the current source cluster is empty, then directly deregister the instances in the target cluster.
|
||||||
|
log.debug("service {} needs to sync ephemeral instance num is null: serviceName ", taskDO.getServiceName());
|
||||||
|
processDeRegisterInstances(taskDO, destNamingService);
|
||||||
|
} else {
|
||||||
|
// Handle batch data synchronization of persistent instances.
|
||||||
|
handlerPersistenceInstance(taskDO, destNamingService, sourceInstances, level);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
syncTaskTap.remove(taskId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void handlerPersistenceInstance(TaskDO taskDO, NamingService destNamingService,
|
||||||
|
List<Instance> sourceInstances, int level) throws NacosException {
|
||||||
|
List<Instance> needRegisterInstance = new ArrayList<>();
|
||||||
|
for (Instance instance : sourceInstances) {
|
||||||
|
if (needSync(instance.getMetadata(), level, taskDO.getDestClusterId())) {
|
||||||
|
needRegisterInstance.add(buildSyncInstance(instance, taskDO));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
List<Instance> destAllInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||||
|
|
||||||
|
// Get the instances that the destination cluster has already synchronized
|
||||||
|
List<Instance> destHasSyncInstances = destAllInstances.stream()
|
||||||
|
.filter(instance -> hasSync(instance, taskDO.getSourceClusterId())).collect(Collectors.toList());
|
||||||
|
|
||||||
|
// The following two conversions are necessary because the Nacos Instance's equals method
|
||||||
|
// is flawed and cannot be used for direct comparison.
|
||||||
|
// The reason is that Instance's equals method compares Metadata using the toString method,
|
||||||
|
// and Metadata is of type HashMap, which does not guarantee order in its toString representation.
|
||||||
|
|
||||||
|
// Convert destHasSyncInstances to a Map with the concatenated string as key
|
||||||
|
Map<String, Instance> destInstanceMap = destHasSyncInstances.stream()
|
||||||
|
.collect(Collectors.toMap(NacosSyncToNacosServiceImpl::getInstanceKey, instance -> instance));
|
||||||
|
|
||||||
|
// Convert newInstances to a Map with the concatenated string as key
|
||||||
|
Map<String, Instance> needRegisterMap = needRegisterInstance.stream()
|
||||||
|
.collect(Collectors.toMap(NacosSyncToNacosServiceImpl::getInstanceKey, instance -> instance));
|
||||||
|
|
||||||
|
// Remove instances from newInstanceMap that are present in destInstanceMap
|
||||||
|
List<Instance> newInstances = removeSyncedInstances(destInstanceMap, needRegisterMap);
|
||||||
|
|
||||||
|
// Remove instances from destInstanceMap that are present in newInstanceMap
|
||||||
|
List<Instance> invalidInstances = getInvalidInstances(destInstanceMap, needRegisterMap);
|
||||||
|
|
||||||
|
// Register each instance one by one. Take one instance at a time.
|
||||||
|
for (Instance newInstance : newInstances) {
|
||||||
|
destNamingService.registerInstance(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
|
newInstance);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (CollectionUtils.isNotEmpty(invalidInstances)) {
|
||||||
|
log.info("taskId: {}, service {} deregistered, number of executions: {}", taskDO.getTaskId(), taskDO.getServiceName(),
|
||||||
|
destHasSyncInstances.size());
|
||||||
|
}
|
||||||
|
for (Instance instance : invalidInstances) {
|
||||||
|
destNamingService.deregisterInstance(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
|
instance);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<Instance> getInvalidInstances(Map<String, Instance> destInstanceMap, Map<String, Instance> needRegisterMap) {
|
||||||
|
Map<String, Instance> destClone = new HashMap<>(destInstanceMap);
|
||||||
|
|
||||||
|
// Convert newInstances to a Map with the concatenated string as key
|
||||||
|
Map<String, Instance> needRegisterClone = new HashMap<>(needRegisterMap);
|
||||||
|
|
||||||
|
// Remove instances from newInstanceMap that are present in destInstanceMap
|
||||||
|
destClone.keySet().removeAll(needRegisterClone.keySet());
|
||||||
|
|
||||||
|
return new ArrayList<>(destClone.values());
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
public List<Instance> removeSyncedInstances(Map<String, Instance> destInstanceMap, Map<String, Instance> needRegisterMap) {
|
||||||
|
// Convert destHasSyncInstances to a Map with the concatenated string as key
|
||||||
|
|
||||||
|
Map<String, Instance> destClone = new HashMap<>(destInstanceMap);
|
||||||
|
|
||||||
|
// Convert newInstances to a Map with the concatenated string as key
|
||||||
|
Map<String, Instance> needRegisterClone = new HashMap<>(needRegisterMap);
|
||||||
|
|
||||||
|
// Remove instances from newInstanceMap that are present in destInstanceMap
|
||||||
|
needRegisterClone.keySet().removeAll(destClone.keySet());
|
||||||
|
|
||||||
|
return new ArrayList<>(needRegisterClone.values());
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
private boolean hasSync(Instance instance, String sourceClusterId) {
|
||||||
|
if (instance.getMetadata() != null) {
|
||||||
|
String sourceClusterKey = instance.getMetadata().get(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY);
|
||||||
|
return sourceClusterKey != null && sourceClusterKey.equals(sourceClusterId);
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* When the number of instances that the source cluster needs to synchronize is 0,
|
||||||
|
* if the target cluster still has instances synchronized with the source cluster,
|
||||||
|
* perform unregistration.
|
||||||
|
*
|
||||||
|
* @param destNamingService Destination cluster naming service
|
||||||
|
* @throws NacosException
|
||||||
|
*/
|
||||||
|
private void processDeRegisterInstances(TaskDO taskDO, NamingService destNamingService) throws NacosException {
|
||||||
|
// If the instances in sourceInstances are empty, it means the instances are offline or do not exist.
|
||||||
|
List<Instance> destInstances = destNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), false);
|
||||||
|
// If the instances in the target cluster are also empty, then no operation is needed.
|
||||||
|
if (CollectionUtils.isEmpty(destInstances)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
destInstances = filterInstancesForRemoval(destInstances, taskDO.getSourceClusterId());
|
||||||
|
if (CollectionUtils.isNotEmpty(destInstances)) {
|
||||||
|
// Deregister each instance one by one. Take one instance at a time.
|
||||||
|
// Need to handle redo, otherwise, it will be registered again.
|
||||||
|
for (Instance destInstance : destInstances) {
|
||||||
|
destNamingService.deregisterInstance(taskDO.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), destInstance);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private String composeInstanceKey(Instance instance) {
|
private List<Instance> filterInstancesForRemoval(List<Instance> destInstances, String sourceClusterId) {
|
||||||
return instance.getIp() + ":" + instance.getPort();
|
return destInstances.stream().filter(instance -> !instance.getMetadata().isEmpty())
|
||||||
|
.filter(instance -> needDeregister(instance.getMetadata().get(SOURCE_CLUSTER_ID_KEY), sourceClusterId))
|
||||||
|
.collect(Collectors.toList());
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean needDeregister(String destClusterId, String sourceClusterId) {
|
||||||
|
if (!StringUtils.isEmpty(destClusterId)) {
|
||||||
|
return destClusterId.equals(sourceClusterId);
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean needSync(Map<String, String> sourceMetaData, int level, String destClusterId) {
|
||||||
|
// Regular cluster (default)
|
||||||
|
if (level == 0) {
|
||||||
|
return SyncService.super.needSync(sourceMetaData);
|
||||||
|
}
|
||||||
|
// Central cluster, as long as the instance is not from the target cluster,
|
||||||
|
// it needs to be synchronized (extended functionality)
|
||||||
|
return !destClusterId.equals(sourceMetaData.get(SOURCE_CLUSTER_ID_KEY));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
public Instance buildSyncInstance(Instance instance, TaskDO taskDO) {
|
private Instance buildSyncInstance(Instance instance, TaskDO taskDO) {
|
||||||
|
Instance temp = getInstance(instance);
|
||||||
|
temp.addMetadata(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||||
|
temp.addMetadata(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||||
|
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||||
|
temp.addMetadata(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||||
|
//The flag is a synchronous instance
|
||||||
|
temp.addMetadata(SkyWalkerConstants.SYNC_INSTANCE_TAG,
|
||||||
|
taskDO.getSourceClusterId() + "@@" + taskDO.getVersion());
|
||||||
|
return temp;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static Instance getInstance(Instance instance) {
|
||||||
Instance temp = new Instance();
|
Instance temp = new Instance();
|
||||||
|
temp.setInstanceId(instance.getInstanceId());
|
||||||
temp.setIp(instance.getIp());
|
temp.setIp(instance.getIp());
|
||||||
temp.setPort(instance.getPort());
|
temp.setPort(instance.getPort());
|
||||||
temp.setClusterName(instance.getClusterName());
|
temp.setClusterName(instance.getClusterName());
|
||||||
|
@ -172,15 +425,19 @@ public class NacosSyncToNacosServiceImpl implements SyncService {
|
||||||
temp.setHealthy(instance.isHealthy());
|
temp.setHealthy(instance.isHealthy());
|
||||||
temp.setWeight(instance.getWeight());
|
temp.setWeight(instance.getWeight());
|
||||||
temp.setEphemeral(instance.isEphemeral());
|
temp.setEphemeral(instance.isEphemeral());
|
||||||
Map<String, String> metaData = new HashMap<>();
|
Map<String, String> metaData = new HashMap<>(instance.getMetadata());
|
||||||
metaData.putAll(instance.getMetadata());
|
|
||||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
|
||||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
|
||||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
|
||||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
|
||||||
temp.setMetadata(metaData);
|
temp.setMetadata(metaData);
|
||||||
return temp;
|
return temp;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static String getInstanceKey(Instance instance) {
|
||||||
|
return String.join("|",
|
||||||
|
instance.getIp(),
|
||||||
|
String.valueOf(instance.getPort()),
|
||||||
|
String.valueOf(instance.getWeight()),
|
||||||
|
String.valueOf(instance.isHealthy()),
|
||||||
|
String.valueOf(instance.isEphemeral()),
|
||||||
|
instance.getClusterName(),
|
||||||
|
instance.getServiceName());
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -12,9 +12,6 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.extension.impl;
|
package com.alibaba.nacossync.extension.impl;
|
||||||
|
|
||||||
import static com.alibaba.nacossync.util.StringUtils.convertDubboFullPathForZk;
|
|
||||||
import static com.alibaba.nacossync.util.StringUtils.convertDubboProvidersPath;
|
|
||||||
|
|
||||||
import com.alibaba.nacos.api.naming.NamingService;
|
import com.alibaba.nacos.api.naming.NamingService;
|
||||||
import com.alibaba.nacos.api.naming.listener.EventListener;
|
import com.alibaba.nacos.api.naming.listener.EventListener;
|
||||||
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
import com.alibaba.nacos.api.naming.listener.NamingEvent;
|
||||||
|
@ -32,20 +29,25 @@ import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.util.DubboConstants;
|
import com.alibaba.nacossync.util.DubboConstants;
|
||||||
import com.google.common.collect.Sets;
|
import com.google.common.collect.Sets;
|
||||||
import java.io.UnsupportedEncodingException;
|
|
||||||
import java.util.HashMap;
|
|
||||||
import java.util.HashSet;
|
|
||||||
import java.util.List;
|
|
||||||
import java.util.Map;
|
|
||||||
import java.util.Set;
|
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.apache.curator.framework.CuratorFramework;
|
import org.apache.curator.framework.CuratorFramework;
|
||||||
import org.apache.curator.framework.recipes.cache.PathChildrenCache;
|
import org.apache.curator.framework.recipes.cache.PathChildrenCache;
|
||||||
import org.apache.curator.framework.recipes.cache.PathChildrenCacheEvent;
|
import org.apache.curator.framework.recipes.cache.PathChildrenCacheEvent;
|
||||||
import org.apache.curator.utils.CloseableUtils;
|
import org.apache.curator.utils.CloseableUtils;
|
||||||
import org.apache.zookeeper.CreateMode;
|
import org.apache.zookeeper.CreateMode;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
|
import java.io.UnsupportedEncodingException;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
|
||||||
|
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||||
|
import static com.alibaba.nacossync.util.StringUtils.convertDubboFullPathForZk;
|
||||||
|
import static com.alibaba.nacossync.util.StringUtils.convertDubboProvidersPath;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Nacos 同步 Zk 数据
|
* Nacos 同步 Zk 数据
|
||||||
|
@ -57,8 +59,7 @@ import org.springframework.beans.factory.annotation.Autowired;
|
||||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.ZK)
|
@NacosSyncService(sourceCluster = ClusterTypeEnum.NACOS, destinationCluster = ClusterTypeEnum.ZK)
|
||||||
public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
|
|
||||||
@Autowired
|
private final MetricsManager metricsManager;
|
||||||
private MetricsManager metricsManager;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @description The Nacos listener map.
|
* @description The Nacos listener map.
|
||||||
|
@ -90,12 +91,13 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
|
|
||||||
private final ZookeeperServerHolder zookeeperServerHolder;
|
private final ZookeeperServerHolder zookeeperServerHolder;
|
||||||
|
|
||||||
@Autowired
|
|
||||||
public NacosSyncToZookeeperServiceImpl(SkyWalkerCacheServices skyWalkerCacheServices,
|
public NacosSyncToZookeeperServiceImpl(SkyWalkerCacheServices skyWalkerCacheServices,
|
||||||
NacosServerHolder nacosServerHolder, ZookeeperServerHolder zookeeperServerHolder) {
|
NacosServerHolder nacosServerHolder, ZookeeperServerHolder zookeeperServerHolder, MetricsManager metricsManager) {
|
||||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
this.nacosServerHolder = nacosServerHolder;
|
this.nacosServerHolder = nacosServerHolder;
|
||||||
this.zookeeperServerHolder = zookeeperServerHolder;
|
this.zookeeperServerHolder = zookeeperServerHolder;
|
||||||
|
this.metricsManager = metricsManager;
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
|
@ -103,15 +105,18 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
try {
|
try {
|
||||||
|
|
||||||
NamingService sourceNamingService =
|
NamingService sourceNamingService =
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||||
EventListener eventListener = nacosListenerMap.remove(taskDO.getTaskId());
|
EventListener eventListener = nacosListenerMap.remove(taskDO.getTaskId());
|
||||||
PathChildrenCache pathChildrenCache = pathChildrenCacheMap.get(taskDO.getTaskId());
|
PathChildrenCache pathChildrenCache = pathChildrenCacheMap.get(taskDO.getTaskId());
|
||||||
sourceNamingService.unsubscribe(taskDO.getServiceName(), eventListener);
|
sourceNamingService.unsubscribe(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
|
eventListener);
|
||||||
CloseableUtils.closeQuietly(pathChildrenCache);
|
CloseableUtils.closeQuietly(pathChildrenCache);
|
||||||
Set<String> instanceUrlSet = instanceBackupMap.get(taskDO.getTaskId());
|
Set<String> instanceUrlSet = instanceBackupMap.get(taskDO.getTaskId());
|
||||||
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId());
|
||||||
for (String instanceUrl : instanceUrlSet) {
|
if(!instanceUrlSet.isEmpty()){
|
||||||
client.delete().quietly().forPath(instanceUrl);
|
for (String instanceUrl : instanceUrlSet) {
|
||||||
|
client.delete().quietly().forPath(instanceUrl);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("delete task from nacos to zk was failed, taskId:{}", taskDO.getTaskId(), e);
|
log.error("delete task from nacos to zk was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||||
|
@ -122,23 +127,24 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean sync(TaskDO taskDO) {
|
public boolean sync(TaskDO taskDO, Integer index) {
|
||||||
try {
|
try {
|
||||||
NamingService sourceNamingService =
|
NamingService sourceNamingService =
|
||||||
nacosServerHolder.get(taskDO.getSourceClusterId(), taskDO.getGroupName());
|
nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||||
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId(), taskDO.getGroupName());
|
CuratorFramework client = zookeeperServerHolder.get(taskDO.getDestClusterId());
|
||||||
nacosListenerMap.putIfAbsent(taskDO.getTaskId(), event -> {
|
nacosListenerMap.putIfAbsent(taskDO.getTaskId(), event -> {
|
||||||
if (event instanceof NamingEvent) {
|
if (event instanceof NamingEvent) {
|
||||||
try {
|
try {
|
||||||
|
|
||||||
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName());
|
List<Instance> sourceInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||||
Set<String> newInstanceUrlSet = getWaitingToAddInstance(taskDO, client, sourceInstances);
|
Set<String> newInstanceUrlSet = getWaitingToAddInstance(taskDO, client, sourceInstances);
|
||||||
|
|
||||||
// 获取之前的备份 删除无效实例
|
// fetch the instance backup
|
||||||
deleteInvalidInstances(taskDO, client, newInstanceUrlSet);
|
deleteInvalidInstances(taskDO, client, newInstanceUrlSet);
|
||||||
// 替换当前备份为最新备份
|
// replace the instance backup
|
||||||
instanceBackupMap.put(taskDO.getTaskId(), newInstanceUrlSet);
|
instanceBackupMap.put(taskDO.getTaskId(), newInstanceUrlSet);
|
||||||
// 尝试恢复因为zk客户端意外断开导致的实例数据
|
// try to compensate for the removed instance
|
||||||
tryToCompensate(taskDO, sourceNamingService, sourceInstances);
|
tryToCompensate(taskDO, sourceNamingService, sourceInstances);
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("event process fail, taskId:{}", taskDO.getTaskId(), e);
|
log.error("event process fail, taskId:{}", taskDO.getTaskId(), e);
|
||||||
|
@ -148,7 +154,8 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
sourceNamingService.subscribe(taskDO.getServiceName(), nacosListenerMap.get(taskDO.getTaskId()));
|
sourceNamingService.subscribe(taskDO.getServiceName(),getGroupNameOrDefault(taskDO.getGroupName()),
|
||||||
|
nacosListenerMap.get(taskDO.getTaskId()));
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("sync task from nacos to zk was failed, taskId:{}", taskDO.getTaskId(), e);
|
log.error("sync task from nacos to zk was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
@ -160,24 +167,41 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
private void tryToCompensate(TaskDO taskDO, NamingService sourceNamingService, List<Instance> sourceInstances) {
|
private void tryToCompensate(TaskDO taskDO, NamingService sourceNamingService, List<Instance> sourceInstances) {
|
||||||
if (!CollectionUtils.isEmpty(sourceInstances)) {
|
if (!CollectionUtils.isEmpty(sourceInstances)) {
|
||||||
final PathChildrenCache pathCache = getPathCache(taskDO);
|
final PathChildrenCache pathCache = getPathCache(taskDO);
|
||||||
if (pathCache.getListenable().size() == 0) { // 防止重复注册
|
// Avoiding re-registration if there is already a listener registered.
|
||||||
pathCache.getListenable().addListener((zkClient, zkEvent) -> {
|
if (pathCache.getListenable().size() == 0) {
|
||||||
if (zkEvent.getType() == PathChildrenCacheEvent.Type.CHILD_REMOVED) {
|
registerCompensationListener(pathCache, taskDO, sourceNamingService);
|
||||||
List<Instance> allInstances =
|
|
||||||
sourceNamingService.getAllInstances(taskDO.getServiceName());
|
|
||||||
for (Instance instance : allInstances) {
|
|
||||||
String instanceUrl = buildSyncInstance(instance, taskDO);
|
|
||||||
String zkInstancePath = zkEvent.getData().getPath();
|
|
||||||
if (zkInstancePath.equals(instanceUrl)) {
|
|
||||||
zkClient.create().creatingParentsIfNeeded().withMode(CreateMode.EPHEMERAL)
|
|
||||||
.forPath(zkInstancePath);
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void registerCompensationListener(PathChildrenCache pathCache, TaskDO taskDO, NamingService sourceNamingService) {
|
||||||
|
pathCache.getListenable().addListener((zkClient, zkEvent) -> {
|
||||||
|
try {
|
||||||
|
if (zkEvent.getType() == PathChildrenCacheEvent.Type.CHILD_REMOVED) {
|
||||||
|
compensateOnChildRemoval(zkClient, zkEvent, sourceNamingService, taskDO);
|
||||||
|
}
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Error processing ZooKeeper event: {}", zkEvent.getType(), e);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private void compensateOnChildRemoval(CuratorFramework zkClient, PathChildrenCacheEvent zkEvent, NamingService sourceNamingService, TaskDO taskDO) {
|
||||||
|
String zkInstancePath = null;
|
||||||
|
try {
|
||||||
|
List<Instance> allInstances = sourceNamingService.getAllInstances(taskDO.getServiceName(), getGroupNameOrDefault(taskDO.getGroupName()));
|
||||||
|
zkInstancePath = zkEvent.getData().getPath();
|
||||||
|
for (Instance instance : allInstances) {
|
||||||
|
String instanceUrl = buildSyncInstance(instance, taskDO);
|
||||||
|
if (zkInstancePath.equals(instanceUrl)) {
|
||||||
|
zkClient.create().creatingParentsIfNeeded().withMode(CreateMode.EPHEMERAL)
|
||||||
|
.forPath(zkInstancePath);
|
||||||
|
log.info("Compensated by re-creating the removed node at path: {}", zkInstancePath);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Failed to compensate for the removed node at path: {}", zkInstancePath, e);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -210,12 +234,11 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
|
|
||||||
protected String buildSyncInstance(Instance instance, TaskDO taskDO) throws UnsupportedEncodingException {
|
protected String buildSyncInstance(Instance instance, TaskDO taskDO) throws UnsupportedEncodingException {
|
||||||
Map<String, String> metaData = new HashMap<>();
|
Map<String, String> metaData = new HashMap<>(instance.getMetadata());
|
||||||
metaData.putAll(instance.getMetadata());
|
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
|
||||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
||||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||||
|
|
||||||
String servicePath = monitorPath.computeIfAbsent(taskDO.getTaskId(),
|
String servicePath = monitorPath.computeIfAbsent(taskDO.getTaskId(),
|
||||||
key -> convertDubboProvidersPath(metaData.get(DubboConstants.INTERFACE_KEY)));
|
key -> convertDubboProvidersPath(metaData.get(DubboConstants.INTERFACE_KEY)));
|
||||||
|
@ -225,16 +248,16 @@ public class NacosSyncToZookeeperServiceImpl implements SyncService {
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 获取zk path child 监听缓存类
|
* fetch zk path cache
|
||||||
*
|
*
|
||||||
* @param taskDO 任务对象
|
* @param taskDO task instance
|
||||||
* @return zk节点操作缓存对象
|
* @return zk path cache
|
||||||
*/
|
*/
|
||||||
private PathChildrenCache getPathCache(TaskDO taskDO) {
|
private PathChildrenCache getPathCache(TaskDO taskDO) {
|
||||||
return pathChildrenCacheMap.computeIfAbsent(taskDO.getTaskId(), (key) -> {
|
return pathChildrenCacheMap.computeIfAbsent(taskDO.getTaskId(), (key) -> {
|
||||||
try {
|
try {
|
||||||
PathChildrenCache pathChildrenCache = new PathChildrenCache(
|
PathChildrenCache pathChildrenCache = new PathChildrenCache(
|
||||||
zookeeperServerHolder.get(taskDO.getDestClusterId(), ""), monitorPath.get(key), false);
|
zookeeperServerHolder.get(taskDO.getDestClusterId()), monitorPath.get(key), false);
|
||||||
pathChildrenCache.start(PathChildrenCache.StartMode.BUILD_INITIAL_CACHE);
|
pathChildrenCache.start(PathChildrenCache.StartMode.BUILD_INITIAL_CACHE);
|
||||||
return pathChildrenCache;
|
return pathChildrenCache;
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
|
|
|
@ -10,22 +10,8 @@
|
||||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
* specific language governing permissions and limitations under the License.
|
* specific language governing permissions and limitations under the License.
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.extension.impl;
|
|
||||||
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.ALL_SERVICE_NAME_PATTERN;
|
package com.alibaba.nacossync.extension.impl;
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_PATH_FORMAT;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_ROOT_PATH;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.GROUP_KEY;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_IP_KEY;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_PORT_KEY;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.INTERFACE_KEY;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.PROTOCOL_KEY;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.VERSION_KEY;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.WEIGHT_KEY;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.ZOOKEEPER_SEPARATOR;
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.createServiceName;
|
|
||||||
import static com.alibaba.nacossync.util.StringUtils.parseIpAndPortString;
|
|
||||||
import static com.alibaba.nacossync.util.StringUtils.parseQueryString;
|
|
||||||
|
|
||||||
import com.alibaba.nacos.api.exception.NacosException;
|
import com.alibaba.nacos.api.exception.NacosException;
|
||||||
import com.alibaba.nacos.api.naming.NamingService;
|
import com.alibaba.nacos.api.naming.NamingService;
|
||||||
|
@ -40,20 +26,38 @@ import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
import com.alibaba.nacossync.extension.holder.ZookeeperServerHolder;
|
import com.alibaba.nacossync.extension.holder.ZookeeperServerHolder;
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import java.util.HashMap;
|
|
||||||
import java.util.List;
|
|
||||||
import java.util.Map;
|
|
||||||
import java.util.Objects;
|
|
||||||
import java.util.Set;
|
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
|
||||||
import java.util.function.Predicate;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.apache.commons.lang3.StringUtils;
|
import org.apache.commons.lang3.StringUtils;
|
||||||
import org.apache.curator.framework.CuratorFramework;
|
import org.apache.curator.framework.CuratorFramework;
|
||||||
import org.apache.curator.framework.recipes.cache.TreeCache;
|
import org.apache.curator.framework.recipes.cache.TreeCache;
|
||||||
import org.apache.curator.framework.recipes.cache.TreeCacheEvent;
|
import org.apache.curator.framework.recipes.cache.TreeCacheEvent;
|
||||||
import org.apache.curator.utils.CloseableUtils;
|
import org.apache.curator.utils.CloseableUtils;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.Collections;
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Objects;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.ALL_SERVICE_NAME_PATTERN;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_PATH_FORMAT;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_ROOT_PATH;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.GROUP_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_IP_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_PORT_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.INTERFACE_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.PROTOCOL_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.VERSION_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.WEIGHT_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.ZOOKEEPER_SEPARATOR;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.createServiceName;
|
||||||
|
import static com.alibaba.nacossync.util.NacosUtils.getGroupNameOrDefault;
|
||||||
|
import static com.alibaba.nacossync.util.StringUtils.parseIpAndPortString;
|
||||||
|
import static com.alibaba.nacossync.util.StringUtils.parseQueryString;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author paderlol
|
* @author paderlol
|
||||||
|
@ -64,17 +68,19 @@ import org.springframework.beans.factory.annotation.Autowired;
|
||||||
@NacosSyncService(sourceCluster = ClusterTypeEnum.ZK, destinationCluster = ClusterTypeEnum.NACOS)
|
@NacosSyncService(sourceCluster = ClusterTypeEnum.ZK, destinationCluster = ClusterTypeEnum.NACOS)
|
||||||
public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
||||||
|
|
||||||
@Autowired
|
private static final String DEFAULT_WEIGHT = "1.0";
|
||||||
private MetricsManager metricsManager;
|
|
||||||
|
private final MetricsManager metricsManager;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Listener cache of Zookeeper format taskId -> PathChildrenCache instance
|
* Listener cache of Zookeeper format taskId -> PathChildrenCache instance
|
||||||
*/
|
*/
|
||||||
private Map<String, TreeCache> treeCacheMap = new ConcurrentHashMap<>();
|
private final Map<String, TreeCache> treeCacheMap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* service name cache
|
* service name cache
|
||||||
*/
|
*/
|
||||||
private Map<String, String> nacosServiceNameMap = new ConcurrentHashMap<>();
|
private final Map<String, String> nacosServiceNameMap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
private final ZookeeperServerHolder zookeeperServerHolder;
|
private final ZookeeperServerHolder zookeeperServerHolder;
|
||||||
|
|
||||||
|
@ -82,40 +88,34 @@ public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
||||||
|
|
||||||
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
|
|
||||||
@Autowired
|
|
||||||
public ZookeeperSyncToNacosServiceImpl(ZookeeperServerHolder zookeeperServerHolder,
|
public ZookeeperSyncToNacosServiceImpl(ZookeeperServerHolder zookeeperServerHolder,
|
||||||
NacosServerHolder nacosServerHolder, SkyWalkerCacheServices skyWalkerCacheServices) {
|
NacosServerHolder nacosServerHolder, SkyWalkerCacheServices skyWalkerCacheServices,
|
||||||
|
MetricsManager metricsManager) {
|
||||||
this.zookeeperServerHolder = zookeeperServerHolder;
|
this.zookeeperServerHolder = zookeeperServerHolder;
|
||||||
this.nacosServerHolder = nacosServerHolder;
|
this.nacosServerHolder = nacosServerHolder;
|
||||||
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
|
this.metricsManager = metricsManager;
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean sync(TaskDO taskDO) {
|
public boolean sync(TaskDO taskDO, Integer index) {
|
||||||
try {
|
try {
|
||||||
if (treeCacheMap.containsKey(taskDO.getTaskId())) {
|
if (treeCacheMap.containsKey(taskDO.getTaskId())) {
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
if (!initializeTreeCache(taskDO)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
TreeCache treeCache = getTreeCache(taskDO);
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
if (destNamingService == null) {
|
||||||
|
logAndRecordSyncError("Failed to obtain NamingService for destination clusterId: {}", taskDO.getDestClusterId(), null);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
// 初次执行任务统一注册所有实例
|
// 初次执行任务统一注册所有实例
|
||||||
registerAllInstances(taskDO, destNamingService);
|
registerAllInstances(taskDO, destNamingService);
|
||||||
//注册ZK监听
|
setupListener(taskDO, destNamingService);
|
||||||
Objects.requireNonNull(treeCache).getListenable().addListener((client, event) -> {
|
|
||||||
try {
|
|
||||||
|
|
||||||
String path = event.getData().getPath();
|
|
||||||
Map<String, String> queryParam = parseQueryString(path);
|
|
||||||
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
|
||||||
processEvent(taskDO, destNamingService, event, path, queryParam);
|
|
||||||
}
|
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("event process from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
|
||||||
}
|
|
||||||
|
|
||||||
});
|
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("sync task from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
log.error("sync task from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||||
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
@ -123,118 +123,96 @@ public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
private boolean initializeTreeCache(TaskDO taskDO) {
|
||||||
private void processEvent(TaskDO taskDO, NamingService destNamingService, TreeCacheEvent event, String path,
|
TreeCache treeCache = getTreeCache(taskDO);
|
||||||
Map<String, String> queryParam) throws NacosException {
|
if (treeCache == null) {
|
||||||
if(!com.alibaba.nacossync.util.StringUtils.isDubboProviderPath(path)) {
|
logAndRecordSyncError("Failed to obtain TreeCache for taskId: {}", taskDO.getTaskId(), null);
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
Map<String, String> ipAndPortParam = parseIpAndPortString(path);
|
|
||||||
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
|
||||||
String serviceName = queryParam.get(INTERFACE_KEY);
|
|
||||||
switch (event.getType()) {
|
|
||||||
case NODE_ADDED:
|
|
||||||
case NODE_UPDATED:
|
|
||||||
|
|
||||||
destNamingService.registerInstance(
|
|
||||||
getServiceNameFromCache(serviceName, queryParam), instance);
|
|
||||||
break;
|
|
||||||
case NODE_REMOVED:
|
|
||||||
|
|
||||||
destNamingService.deregisterInstance(
|
|
||||||
getServiceNameFromCache(serviceName, queryParam),
|
|
||||||
ipAndPortParam.get(INSTANCE_IP_KEY),
|
|
||||||
Integer.parseInt(ipAndPortParam.get(INSTANCE_PORT_KEY)));
|
|
||||||
nacosServiceNameMap.remove(serviceName);
|
|
||||||
break;
|
|
||||||
default:
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private void registerAllInstances(TaskDO taskDO, NamingService destNamingService) throws Exception {
|
|
||||||
CuratorFramework zk = zookeeperServerHolder.get(taskDO.getSourceClusterId(), "");
|
|
||||||
if(!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
|
||||||
registerALLInstances0(taskDO, destNamingService, zk, taskDO.getServiceName());
|
|
||||||
} else {
|
|
||||||
// 同步全部
|
|
||||||
List<String> serviceList = zk.getChildren().forPath(DUBBO_ROOT_PATH);
|
|
||||||
for(String serviceName : serviceList) {
|
|
||||||
registerALLInstances0(taskDO, destNamingService, zk, serviceName);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private void registerALLInstances0(TaskDO taskDO, NamingService destNamingService, CuratorFramework zk,
|
|
||||||
String serviceName) throws Exception {
|
|
||||||
String path = String.format(DUBBO_PATH_FORMAT, serviceName);
|
|
||||||
if(zk.getChildren()==null) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
List<String> providers = zk.getChildren().forPath(path);
|
|
||||||
for(String provider : providers) {
|
|
||||||
Map<String, String> queryParam = parseQueryString(provider);
|
|
||||||
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
|
||||||
Map<String, String> ipAndPortParam = parseIpAndPortString(path + ZOOKEEPER_SEPARATOR + provider);
|
|
||||||
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
|
||||||
destNamingService.registerInstance(getServiceNameFromCache(serviceName, queryParam),
|
|
||||||
instance);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public boolean delete(TaskDO taskDO) {
|
|
||||||
if (taskDO.getServiceName() == null) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
|
|
||||||
CloseableUtils.closeQuietly(treeCacheMap.get(taskDO.getTaskId()));
|
|
||||||
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId(), null);
|
|
||||||
if(!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
|
||||||
if (nacosServiceNameMap.containsKey(taskDO.getServiceName())) {
|
|
||||||
List<Instance> allInstances =
|
|
||||||
destNamingService.getAllInstances(nacosServiceNameMap.get(taskDO.getServiceName()));
|
|
||||||
for (Instance instance : allInstances) {
|
|
||||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
|
||||||
destNamingService.deregisterInstance(instance.getServiceName(), instance.getIp(),
|
|
||||||
instance.getPort());
|
|
||||||
}
|
|
||||||
nacosServiceNameMap.remove(taskDO.getServiceName());
|
|
||||||
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
Set<String> serviceNames = nacosServiceNameMap.keySet();
|
|
||||||
for(String serviceName : serviceNames) {
|
|
||||||
|
|
||||||
if (nacosServiceNameMap.containsKey(serviceName)) {
|
|
||||||
List<Instance> allInstances =
|
|
||||||
destNamingService.getAllInstances(serviceName);
|
|
||||||
for (Instance instance : allInstances) {
|
|
||||||
if (needDelete(instance.getMetadata(), taskDO)) {
|
|
||||||
destNamingService.deregisterInstance(instance.getServiceName(), instance.getIp(),
|
|
||||||
instance.getPort());
|
|
||||||
}
|
|
||||||
nacosServiceNameMap.remove(serviceName);
|
|
||||||
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
} catch (Exception e) {
|
|
||||||
log.error("delete task from zookeeper to nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
|
||||||
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
private void logAndRecordSyncError(String message, String taskId, Exception e) {
|
||||||
|
if (e != null) {
|
||||||
|
log.error(message, taskId, e);
|
||||||
|
} else {
|
||||||
|
log.error(message, taskId);
|
||||||
|
}
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.SYNC_ERROR);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void setupListener(TaskDO taskDO, NamingService destNamingService) {
|
||||||
|
TreeCache treeCache = Objects.requireNonNull(getTreeCache(taskDO));
|
||||||
|
treeCache.getListenable().addListener((client, event) -> {
|
||||||
|
try {
|
||||||
|
// INITIALIZED is a special event that is not triggered by the Zookeeper server
|
||||||
|
if(event.getData()==null){
|
||||||
|
log.warn("TreeCache event data is null, taskId:{}", taskDO.getTaskId());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
String path = event.getData().getPath();
|
||||||
|
Map<String, String> queryParam = parseQueryString(path);
|
||||||
|
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
||||||
|
processEvent(taskDO, destNamingService, event, path, queryParam);
|
||||||
|
}
|
||||||
|
} catch (Exception e) {
|
||||||
|
logAndRecordSyncError("Event process from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
public boolean delete(TaskDO taskDO) {
|
||||||
|
if (taskDO.getServiceName() == null) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
CloseableUtils.closeQuietly(treeCacheMap.get(taskDO.getTaskId()));
|
||||||
|
|
||||||
|
try {
|
||||||
|
NamingService destNamingService = nacosServerHolder.get(taskDO.getDestClusterId());
|
||||||
|
if (destNamingService == null) {
|
||||||
|
log.error("Failed to obtain NamingService for destination clusterId: {}", taskDO.getDestClusterId());
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
Set<String> serviceNames = getServiceNamesToDelete(taskDO);
|
||||||
|
deleteInstances(serviceNames, destNamingService, taskDO);
|
||||||
|
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Delete task from Zookeeper to Nacos was failed, taskId:{}", taskDO.getTaskId(), e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.DELETE_ERROR);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private Set<String> getServiceNamesToDelete(TaskDO taskDO) {
|
||||||
|
if (!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
||||||
|
return new HashSet<>(Collections.singleton(taskDO.getServiceName()));
|
||||||
|
} else {
|
||||||
|
return new HashSet<>(nacosServiceNameMap.keySet());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void deleteInstances(Set<String> serviceNames, NamingService destNamingService, TaskDO taskDO) {
|
||||||
|
for (String serviceName : serviceNames) {
|
||||||
|
try {
|
||||||
|
List<Instance> allInstances = destNamingService.getAllInstances(serviceName,
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), new ArrayList<>(), true);
|
||||||
|
for (Instance instance : allInstances) {
|
||||||
|
if (needDelete(instance.getMetadata(), taskDO)) {
|
||||||
|
destNamingService.deregisterInstance(instance.getServiceName(),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), instance.getIp(), instance.getPort());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
nacosServiceNameMap.remove(serviceName);
|
||||||
|
} catch (NacosException e) {
|
||||||
|
log.error("Failed to deregister service instance for serviceName: {}", serviceName, e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* fetch the Path cache when the task sync
|
* fetch the Path cache when the task sync
|
||||||
|
@ -242,9 +220,8 @@ public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
||||||
protected TreeCache getTreeCache(TaskDO taskDO) {
|
protected TreeCache getTreeCache(TaskDO taskDO) {
|
||||||
return treeCacheMap.computeIfAbsent(taskDO.getTaskId(), (key) -> {
|
return treeCacheMap.computeIfAbsent(taskDO.getTaskId(), (key) -> {
|
||||||
try {
|
try {
|
||||||
TreeCache treeCache =
|
TreeCache treeCache = new TreeCache(zookeeperServerHolder.get(taskDO.getSourceClusterId()),
|
||||||
new TreeCache(zookeeperServerHolder.get(taskDO.getSourceClusterId(), ""),
|
DUBBO_ROOT_PATH);
|
||||||
DUBBO_ROOT_PATH);
|
|
||||||
treeCache.start();
|
treeCache.start();
|
||||||
return treeCache;
|
return treeCache;
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
|
@ -256,47 +233,144 @@ public class ZookeeperSyncToNacosServiceImpl implements SyncService {
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The instance information that needs to be synchronized is matched based on the dubbo version and the grouping
|
* Checks if the instance information needs to be synchronized based on the dubbo version, grouping name,
|
||||||
* name
|
* and service name.
|
||||||
*/
|
*/
|
||||||
protected boolean isMatch(TaskDO taskDO, Map<String, String> queryParam) {
|
protected boolean isMatch(TaskDO taskDO, Map<String, String> queryParam) {
|
||||||
Predicate<TaskDO> isVersionEq = (task) -> StringUtils.isBlank(taskDO.getVersion())
|
return isVersionMatch(taskDO, queryParam) &&
|
||||||
|| StringUtils.equals(task.getVersion(), queryParam.get(VERSION_KEY));
|
isGroupMatch(taskDO, queryParam) &&
|
||||||
Predicate<TaskDO> isGroupEq = (task) -> StringUtils.isBlank(taskDO.getGroupName())
|
isServiceMatch(taskDO, queryParam) ||
|
||||||
|| StringUtils.equals(task.getGroupName(), queryParam.get(GROUP_KEY));
|
isMatchAllServices(taskDO);
|
||||||
return isVersionEq.and(isGroupEq).test(taskDO);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private boolean isVersionMatch(TaskDO task, Map<String, String> queryParam) {
|
||||||
|
return StringUtils.isBlank(task.getVersion()) || StringUtils.equals(task.getVersion(), queryParam.get(VERSION_KEY));
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isGroupMatch(TaskDO task, Map<String, String> queryParam) {
|
||||||
|
return StringUtils.isBlank(task.getGroupName()) || StringUtils.equals(task.getGroupName(), queryParam.get(GROUP_KEY));
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isServiceMatch(TaskDO task, Map<String, String> queryParam) {
|
||||||
|
return StringUtils.isNotBlank(task.getServiceName()) && StringUtils.equals(task.getServiceName(), queryParam.get(INTERFACE_KEY));
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isMatchAllServices(TaskDO task) {
|
||||||
|
return StringUtils.isNotBlank(task.getServiceName()) && StringUtils.equals(task.getServiceName(), ALL_SERVICE_NAME_PATTERN);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* create Nacos service instance
|
* Builds a synchronized Nacos instance from Zookeeper data.
|
||||||
*
|
*
|
||||||
* @param queryParam dubbo metadata
|
* @param queryParam Parameters obtained from the query string.
|
||||||
* @param ipAndPortMap dubbo ip and address
|
* @param ipAndPortMap IP and port information.
|
||||||
|
* @param taskDO Task details.
|
||||||
|
* @return A fully configured Nacos instance.
|
||||||
*/
|
*/
|
||||||
protected Instance buildSyncInstance(Map<String, String> queryParam, Map<String, String> ipAndPortMap,
|
protected Instance buildSyncInstance(Map<String, String> queryParam, Map<String, String> ipAndPortMap, TaskDO taskDO) {
|
||||||
TaskDO taskDO) {
|
Instance instance = new Instance();
|
||||||
Instance temp = new Instance();
|
instance.setIp(ipAndPortMap.get(INSTANCE_IP_KEY));
|
||||||
temp.setIp(ipAndPortMap.get(INSTANCE_IP_KEY));
|
instance.setPort(Integer.parseInt(ipAndPortMap.get(INSTANCE_PORT_KEY)));
|
||||||
temp.setPort(Integer.parseInt(ipAndPortMap.get(INSTANCE_PORT_KEY)));
|
instance.setServiceName(getServiceNameFromCache(taskDO.getTaskId(), queryParam));
|
||||||
temp.setServiceName(getServiceNameFromCache(taskDO.getTaskId(), queryParam));
|
instance.setWeight(parseWeight(queryParam));
|
||||||
temp.setWeight(Double.parseDouble(queryParam.get(WEIGHT_KEY) == null ? "1.0" : queryParam.get(WEIGHT_KEY)));
|
instance.setHealthy(true);
|
||||||
temp.setHealthy(true);
|
instance.setMetadata(buildMetadata(queryParam, ipAndPortMap, taskDO));
|
||||||
|
return instance;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
private double parseWeight(Map<String, String> queryParam) {
|
||||||
|
try {
|
||||||
|
return Double.parseDouble(queryParam.getOrDefault(WEIGHT_KEY, DEFAULT_WEIGHT));
|
||||||
|
} catch (NumberFormatException e) {
|
||||||
|
log.error("Error parsing weight: {}", queryParam.get(WEIGHT_KEY), e);
|
||||||
|
return Double.parseDouble(DEFAULT_WEIGHT); // Default weight in case of error
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private Map<String, String> buildMetadata(Map<String, String> queryParam, Map<String, String> ipAndPortMap, TaskDO taskDO) {
|
||||||
Map<String, String> metaData = new HashMap<>(queryParam);
|
Map<String, String> metaData = new HashMap<>(queryParam);
|
||||||
metaData.put(PROTOCOL_KEY, ipAndPortMap.get(PROTOCOL_KEY));
|
metaData.put(PROTOCOL_KEY, ipAndPortMap.get(PROTOCOL_KEY));
|
||||||
metaData.put(SkyWalkerConstants.DEST_CLUSTERID_KEY, taskDO.getDestClusterId());
|
metaData.put(SkyWalkerConstants.DEST_CLUSTER_ID_KEY, taskDO.getDestClusterId());
|
||||||
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY,
|
metaData.put(SkyWalkerConstants.SYNC_SOURCE_KEY, skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
||||||
skyWalkerCacheServices.getClusterType(taskDO.getSourceClusterId()).getCode());
|
metaData.put(SkyWalkerConstants.SOURCE_CLUSTER_ID_KEY, taskDO.getSourceClusterId());
|
||||||
metaData.put(SkyWalkerConstants.SOURCE_CLUSTERID_KEY, taskDO.getSourceClusterId());
|
return metaData;
|
||||||
temp.setMetadata(metaData);
|
|
||||||
return temp;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private void processEvent(TaskDO taskDO, NamingService destNamingService, TreeCacheEvent event, String path,
|
||||||
|
Map<String, String> queryParam) throws NacosException {
|
||||||
|
if (!com.alibaba.nacossync.util.StringUtils.isDubboProviderPath(path)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
Map<String, String> ipAndPortParam = parseIpAndPortString(path);
|
||||||
|
if (ipAndPortParam.isEmpty()) {
|
||||||
|
log.error("Invalid IP and Port data extracted from path: {}", path);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
||||||
|
String serviceName = queryParam.get(INTERFACE_KEY);
|
||||||
|
if (serviceName == null || serviceName.isEmpty()) {
|
||||||
|
log.error("Service name is missing in the query parameters.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
switch (event.getType()) {
|
||||||
|
case NODE_ADDED:
|
||||||
|
case NODE_UPDATED:
|
||||||
|
|
||||||
|
destNamingService.registerInstance(getServiceNameFromCache(serviceName, queryParam),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||||
|
break;
|
||||||
|
case NODE_REMOVED:
|
||||||
|
|
||||||
|
destNamingService.deregisterInstance(getServiceNameFromCache(serviceName, queryParam),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), ipAndPortParam.get(INSTANCE_IP_KEY),
|
||||||
|
Integer.parseInt(ipAndPortParam.get(INSTANCE_PORT_KEY)));
|
||||||
|
nacosServiceNameMap.remove(serviceName);
|
||||||
|
break;
|
||||||
|
default:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void registerAllInstances(TaskDO taskDO, NamingService destNamingService) throws Exception {
|
||||||
|
CuratorFramework zk = zookeeperServerHolder.get(taskDO.getSourceClusterId());
|
||||||
|
if (!ALL_SERVICE_NAME_PATTERN.equals(taskDO.getServiceName())) {
|
||||||
|
registerALLInstances0(taskDO, destNamingService, zk, taskDO.getServiceName());
|
||||||
|
} else {
|
||||||
|
// 同步全部
|
||||||
|
List<String> serviceList = zk.getChildren().forPath(DUBBO_ROOT_PATH);
|
||||||
|
for (String serviceName : serviceList) {
|
||||||
|
registerALLInstances0(taskDO, destNamingService, zk, serviceName);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void registerALLInstances0(TaskDO taskDO, NamingService destNamingService, CuratorFramework zk,
|
||||||
|
String serviceName) throws Exception {
|
||||||
|
String path = String.format(DUBBO_PATH_FORMAT, serviceName);
|
||||||
|
if (zk.getChildren() == null) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
List<String> providers = zk.getChildren().forPath(path);
|
||||||
|
for (String provider : providers) {
|
||||||
|
Map<String, String> queryParam = parseQueryString(provider);
|
||||||
|
if (isMatch(taskDO, queryParam) && needSync(queryParam)) {
|
||||||
|
Map<String, String> ipAndPortParam = parseIpAndPortString(path + ZOOKEEPER_SEPARATOR + provider);
|
||||||
|
Instance instance = buildSyncInstance(queryParam, ipAndPortParam, taskDO);
|
||||||
|
destNamingService.registerInstance(getServiceNameFromCache(serviceName, queryParam),
|
||||||
|
getGroupNameOrDefault(taskDO.getGroupName()), instance);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* cteate Dubbo service name
|
* cteate Dubbo service name
|
||||||
*
|
*
|
||||||
* @param serviceName dubbo service name
|
* @param serviceName dubbo service name
|
||||||
* @param queryParam dubbo metadata
|
* @param queryParam dubbo metadata
|
||||||
*/
|
*/
|
||||||
protected String getServiceNameFromCache(String serviceName, Map<String, String> queryParam) {
|
protected String getServiceNameFromCache(String serviceName, Map<String, String> queryParam) {
|
||||||
return nacosServiceNameMap.computeIfAbsent(serviceName, (key) -> createServiceName(queryParam));
|
return nacosServiceNameMap.computeIfAbsent(serviceName, (key) -> createServiceName(queryParam));
|
||||||
|
|
|
@ -23,14 +23,18 @@ import java.util.concurrent.TimeUnit;
|
||||||
@Service
|
@Service
|
||||||
public class MetricsManager implements CommandLineRunner {
|
public class MetricsManager implements CommandLineRunner {
|
||||||
|
|
||||||
@Autowired
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
|
||||||
|
|
||||||
@Autowired
|
private final ClusterAccessService clusterAccessService;
|
||||||
private ClusterAccessService clusterAccessService;
|
|
||||||
|
|
||||||
@Autowired
|
private final TaskAccessService taskAccessService;
|
||||||
private TaskAccessService taskAccessService;
|
|
||||||
|
public MetricsManager(SkyWalkerCacheServices skyWalkerCacheServices, ClusterAccessService clusterAccessService,
|
||||||
|
TaskAccessService taskAccessService) {
|
||||||
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Callback used to run the bean.
|
* Callback used to run the bean.
|
||||||
|
|
|
@ -17,17 +17,28 @@
|
||||||
package com.alibaba.nacossync.pojo.model;
|
package com.alibaba.nacossync.pojo.model;
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
|
import lombok.Getter;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import lombok.Setter;
|
||||||
|
import lombok.ToString;
|
||||||
|
import org.hibernate.proxy.HibernateProxy;
|
||||||
|
|
||||||
|
import javax.persistence.Entity;
|
||||||
|
import javax.persistence.GeneratedValue;
|
||||||
|
import javax.persistence.GenerationType;
|
||||||
|
import javax.persistence.Id;
|
||||||
|
import javax.persistence.Table;
|
||||||
import java.io.Serializable;
|
import java.io.Serializable;
|
||||||
|
import java.util.Objects;
|
||||||
import javax.persistence.*;
|
|
||||||
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: EnvDO.java, v 0.1 2018-09-25 PM 4:17 NacosSync Exp $$
|
* @version $Id: EnvDO.java, v 0.1 2018-09-25 PM 4:17 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
@Data
|
@Getter
|
||||||
|
@Setter
|
||||||
|
@ToString
|
||||||
|
@RequiredArgsConstructor
|
||||||
@Entity
|
@Entity
|
||||||
@Table(name = "cluster")
|
@Table(name = "cluster")
|
||||||
public class ClusterDO implements Serializable {
|
public class ClusterDO implements Serializable {
|
||||||
|
@ -66,4 +77,34 @@ public class ClusterDO implements Serializable {
|
||||||
*/
|
*/
|
||||||
private String password;
|
private String password;
|
||||||
|
|
||||||
|
private String namespace;
|
||||||
|
|
||||||
|
private Integer clusterLevel;
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public final boolean equals(Object o) {
|
||||||
|
if (this == o) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (o == null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
Class<?> oEffectiveClass =
|
||||||
|
o instanceof HibernateProxy ? ((HibernateProxy) o).getHibernateLazyInitializer().getPersistentClass()
|
||||||
|
: o.getClass();
|
||||||
|
Class<?> thisEffectiveClass =
|
||||||
|
this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||||
|
.getPersistentClass() : this.getClass();
|
||||||
|
if (thisEffectiveClass != oEffectiveClass) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
ClusterDO clusterDO = (ClusterDO) o;
|
||||||
|
return getId() != null && Objects.equals(getId(), clusterDO.getId());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public final int hashCode() {
|
||||||
|
return this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||||
|
.getPersistentClass().hashCode() : getClass().hashCode();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -16,15 +16,28 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.pojo.model;
|
package com.alibaba.nacossync.pojo.model;
|
||||||
|
|
||||||
import javax.persistence.*;
|
|
||||||
|
|
||||||
import lombok.Data;
|
import lombok.Getter;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import lombok.Setter;
|
||||||
|
import lombok.ToString;
|
||||||
|
import org.hibernate.proxy.HibernateProxy;
|
||||||
|
|
||||||
|
import javax.persistence.Entity;
|
||||||
|
import javax.persistence.GeneratedValue;
|
||||||
|
import javax.persistence.GenerationType;
|
||||||
|
import javax.persistence.Id;
|
||||||
|
import javax.persistence.Table;
|
||||||
|
import java.util.Objects;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: SystemConfig.java, v 0.1 2018-09-26 上午1:48 NacosSync Exp $$
|
* @version $Id: SystemConfig.java, v 0.1 2018-09-26 上午1:48 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
@Data
|
@Getter
|
||||||
|
@Setter
|
||||||
|
@ToString
|
||||||
|
@RequiredArgsConstructor
|
||||||
@Entity
|
@Entity
|
||||||
@Table(name = "system_config")
|
@Table(name = "system_config")
|
||||||
public class SystemConfigDO {
|
public class SystemConfigDO {
|
||||||
|
@ -35,4 +48,30 @@ public class SystemConfigDO {
|
||||||
private String configValue;
|
private String configValue;
|
||||||
private String configDesc;
|
private String configDesc;
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public final boolean equals(Object o) {
|
||||||
|
if (this == o) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (o == null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
Class<?> oEffectiveClass =
|
||||||
|
o instanceof HibernateProxy ? ((HibernateProxy) o).getHibernateLazyInitializer().getPersistentClass()
|
||||||
|
: o.getClass();
|
||||||
|
Class<?> thisEffectiveClass =
|
||||||
|
this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||||
|
.getPersistentClass() : this.getClass();
|
||||||
|
if (thisEffectiveClass != oEffectiveClass) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
SystemConfigDO that = (SystemConfigDO) o;
|
||||||
|
return getId() != null && Objects.equals(getId(), that.getId());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public final int hashCode() {
|
||||||
|
return this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||||
|
.getPersistentClass().hashCode() : getClass().hashCode();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -12,16 +12,29 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.pojo.model;
|
package com.alibaba.nacossync.pojo.model;
|
||||||
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
import javax.persistence.*;
|
import lombok.Getter;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import lombok.Setter;
|
||||||
|
import lombok.ToString;
|
||||||
|
import org.hibernate.proxy.HibernateProxy;
|
||||||
|
|
||||||
|
import javax.persistence.Entity;
|
||||||
|
import javax.persistence.GeneratedValue;
|
||||||
|
import javax.persistence.GenerationType;
|
||||||
|
import javax.persistence.Id;
|
||||||
|
import javax.persistence.Table;
|
||||||
import java.io.Serializable;
|
import java.io.Serializable;
|
||||||
|
import java.util.Objects;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: TaskDo.java, v 0.1 2018-09-24 PM11:53 NacosSync Exp $$
|
* @version $Id: TaskDo.java, v 0.1 2018-09-24 PM11:53 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
@Data
|
@Getter
|
||||||
|
@Setter
|
||||||
|
@ToString
|
||||||
|
@RequiredArgsConstructor
|
||||||
@Entity
|
@Entity
|
||||||
@Table(name = "task")
|
@Table(name = "task")
|
||||||
public class TaskDO implements Serializable {
|
public class TaskDO implements Serializable {
|
||||||
|
@ -70,4 +83,31 @@ public class TaskDO implements Serializable {
|
||||||
* operation id,The operation id follow when the task status changes
|
* operation id,The operation id follow when the task status changes
|
||||||
*/
|
*/
|
||||||
private String operationId;
|
private String operationId;
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public final boolean equals(Object o) {
|
||||||
|
if (this == o) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (o == null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
Class<?> oEffectiveClass =
|
||||||
|
o instanceof HibernateProxy ? ((HibernateProxy) o).getHibernateLazyInitializer().getPersistentClass()
|
||||||
|
: o.getClass();
|
||||||
|
Class<?> thisEffectiveClass =
|
||||||
|
this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||||
|
.getPersistentClass() : this.getClass();
|
||||||
|
if (thisEffectiveClass != oEffectiveClass) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
TaskDO taskDO = (TaskDO) o;
|
||||||
|
return getId() != null && Objects.equals(getId(), taskDO.getId());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public final int hashCode() {
|
||||||
|
return this instanceof HibernateProxy ? ((HibernateProxy) this).getHibernateLazyInitializer()
|
||||||
|
.getPersistentClass().hashCode() : getClass().hashCode();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -17,10 +17,10 @@
|
||||||
package com.alibaba.nacossync.pojo.request;
|
package com.alibaba.nacossync.pojo.request;
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
import lombok.Data;
|
import lombok.Data;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: AddClusterRequest.java, v 0.1 2018-09-25 PM 10:27 NacosSync Exp $$
|
* @version $Id: AddClusterRequest.java, v 0.1 2018-09-25 PM 10:27 NacosSync Exp $$
|
||||||
|
@ -52,5 +52,6 @@ public class ClusterAddRequest extends BaseRequest {
|
||||||
* The password of the Nacos.
|
* The password of the Nacos.
|
||||||
*/
|
*/
|
||||||
private String password;
|
private String password;
|
||||||
|
private String namespace;
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -0,0 +1,44 @@
|
||||||
|
/*
|
||||||
|
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||||
|
* contributor license agreements. See the NOTICE file distributed with
|
||||||
|
* this work for additional information regarding copyright ownership.
|
||||||
|
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||||
|
* (the "License"); you may not use this file except in compliance with
|
||||||
|
* the License. You may obtain a copy of the License at
|
||||||
|
*
|
||||||
|
* http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
*
|
||||||
|
* Unless required by applicable law or agreed to in writing, software
|
||||||
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
* See the License for the specific language governing permissions and
|
||||||
|
* limitations under the License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package com.alibaba.nacossync.pojo.request;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @author NacosSync
|
||||||
|
* @version $Id: TaskAddAllRequest.java, v 0.1 2022-03-23 AM12:13 NacosSync Exp $$
|
||||||
|
*/
|
||||||
|
@Data
|
||||||
|
public class TaskAddAllRequest extends BaseRequest {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* eg: b7bacb110199d5bb83b9757038fadeb0 .
|
||||||
|
*/
|
||||||
|
private String sourceClusterId;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* eg: bbdad57833a0e4f0981f6f3349005617 .
|
||||||
|
*/
|
||||||
|
private String destClusterId;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* whether to exclude subscriber.
|
||||||
|
*/
|
||||||
|
private boolean excludeConsumer = true;
|
||||||
|
|
||||||
|
}
|
|
@ -14,13 +14,15 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.pojo.view;
|
package com.alibaba.nacossync.pojo.view;
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import java.io.Serializable;
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
|
|
||||||
import lombok.Data;
|
import lombok.Data;
|
||||||
|
|
||||||
|
import java.io.Serializable;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: ClusterModel.java, v 0.1 2018-09-25 下午11:09 NacosSync Exp $$
|
* @version $Id: ClusterModel.java, v 0.1 2018-09-25 下午11:09 NacosSync Exp $$
|
||||||
|
@ -29,18 +31,36 @@ import lombok.Data;
|
||||||
public class ClusterModel implements Serializable {
|
public class ClusterModel implements Serializable {
|
||||||
|
|
||||||
private String clusterId;
|
private String clusterId;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* json format,["192.168.1:8080","192.168.2?key=1"]
|
* json format,["192.168.1:8080","192.168.2?key=1"]
|
||||||
*/
|
*/
|
||||||
private String connectKeyList;
|
private String connectKeyList;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* cluster name, eg:cluster of ShangHai(edas-sh)
|
* cluster name, eg:cluster of ShangHai(edas-sh)
|
||||||
*/
|
*/
|
||||||
private String clusterName;
|
private String clusterName;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* cluster type, eg cluster of CS,cluster of Nacos,
|
* cluster type, eg cluster of CS,cluster of Nacos,
|
||||||
*
|
*
|
||||||
* @see ClusterTypeEnum
|
* @see ClusterTypeEnum
|
||||||
*/
|
*/
|
||||||
private String clusterType;
|
private String clusterType;
|
||||||
|
|
||||||
|
private String namespace;
|
||||||
|
|
||||||
|
private String userName;
|
||||||
|
|
||||||
|
public static ClusterModel from(ClusterDO clusterDO) {
|
||||||
|
ClusterModel clusterModel = new ClusterModel();
|
||||||
|
clusterModel.setClusterId(clusterDO.getClusterId());
|
||||||
|
clusterModel.setConnectKeyList(clusterDO.getConnectKeyList());
|
||||||
|
clusterModel.setClusterType(clusterDO.getClusterType());
|
||||||
|
clusterModel.setClusterName(clusterDO.getClusterName());
|
||||||
|
clusterModel.setNamespace(clusterDO.getNamespace());
|
||||||
|
clusterModel.setUserName(clusterDO.getUserName());
|
||||||
|
return clusterModel;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,8 +14,10 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.pojo.view;
|
package com.alibaba.nacossync.pojo.view;
|
||||||
|
|
||||||
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import lombok.Data;
|
import lombok.Data;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -26,10 +28,25 @@ import lombok.Data;
|
||||||
public class TaskModel {
|
public class TaskModel {
|
||||||
|
|
||||||
private String taskId;
|
private String taskId;
|
||||||
|
|
||||||
private String sourceClusterId;
|
private String sourceClusterId;
|
||||||
|
|
||||||
private String destClusterId;
|
private String destClusterId;
|
||||||
|
|
||||||
private String serviceName;
|
private String serviceName;
|
||||||
|
|
||||||
private String groupName;
|
private String groupName;
|
||||||
|
|
||||||
private String taskStatus;
|
private String taskStatus;
|
||||||
|
|
||||||
|
public static TaskModel from(TaskDO taskDO) {
|
||||||
|
TaskModel taskModel = new TaskModel();
|
||||||
|
taskModel.setDestClusterId(taskDO.getDestClusterId());
|
||||||
|
taskModel.setGroupName(taskDO.getGroupName());
|
||||||
|
taskModel.setServiceName(taskDO.getServiceName());
|
||||||
|
taskModel.setSourceClusterId(taskDO.getSourceClusterId());
|
||||||
|
taskModel.setTaskStatus(taskDO.getTaskStatus());
|
||||||
|
taskModel.setTaskId(taskDO.getTaskId());
|
||||||
|
return taskModel;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -45,8 +45,7 @@ public class SkyWalkerTemplate {
|
||||||
}
|
}
|
||||||
|
|
||||||
private static <T extends BaseResult> void initExceptionResult(T result, Throwable e) {
|
private static <T extends BaseResult> void initExceptionResult(T result, Throwable e) {
|
||||||
if (e instanceof SkyWalkerException) {
|
if (e instanceof SkyWalkerException skyWalkerException) {
|
||||||
SkyWalkerException skyWalkerException = (SkyWalkerException) e;
|
|
||||||
if (null != skyWalkerException.getResultCode()) {
|
if (null != skyWalkerException.getResultCode()) {
|
||||||
result.setResultCode(skyWalkerException.getResultCode().getCode());
|
result.setResultCode(skyWalkerException.getResultCode().getCode());
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,21 +14,20 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import com.alibaba.fastjson.JSONObject;
|
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
|
||||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
import com.alibaba.nacossync.pojo.request.ClusterAddRequest;
|
import com.alibaba.nacossync.pojo.request.ClusterAddRequest;
|
||||||
import com.alibaba.nacossync.pojo.result.ClusterAddResult;
|
import com.alibaba.nacossync.pojo.result.ClusterAddResult;
|
||||||
import com.alibaba.nacossync.template.Processor;
|
import com.alibaba.nacossync.template.Processor;
|
||||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||||
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.apache.commons.lang3.StringUtils;
|
import org.apache.commons.lang3.StringUtils;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -39,24 +38,28 @@ import org.springframework.stereotype.Service;
|
||||||
@Service
|
@Service
|
||||||
public class ClusterAddProcessor implements Processor<ClusterAddRequest, ClusterAddResult> {
|
public class ClusterAddProcessor implements Processor<ClusterAddRequest, ClusterAddResult> {
|
||||||
|
|
||||||
@Autowired
|
|
||||||
private MetricsManager metricsManager;
|
|
||||||
|
|
||||||
@Autowired
|
private final ClusterAccessService clusterAccessService;
|
||||||
private ClusterAccessService clusterAccessService;
|
|
||||||
|
private final ObjectMapper objectMapper;
|
||||||
|
|
||||||
|
public ClusterAddProcessor(ClusterAccessService clusterAccessService, ObjectMapper objectMapper) {
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
this.objectMapper = objectMapper;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(ClusterAddRequest clusterAddRequest, ClusterAddResult clusterAddResult,
|
public void process(ClusterAddRequest clusterAddRequest, ClusterAddResult clusterAddResult, Object... others)
|
||||||
Object... others) throws Exception {
|
throws Exception {
|
||||||
ClusterDO clusterDO = new ClusterDO();
|
ClusterDO clusterDO = new ClusterDO();
|
||||||
|
|
||||||
if (null == clusterAddRequest.getConnectKeyList() || 0 == clusterAddRequest.getConnectKeyList().size()) {
|
if (null == clusterAddRequest.getConnectKeyList() || clusterAddRequest.getConnectKeyList().isEmpty()) {
|
||||||
|
|
||||||
throw new SkyWalkerException("集群列表不能为空!");
|
throw new SkyWalkerException("集群列表不能为空!");
|
||||||
}
|
}
|
||||||
|
|
||||||
if (StringUtils.isBlank(clusterAddRequest.getClusterName()) || StringUtils
|
if (StringUtils.isBlank(clusterAddRequest.getClusterName()) || StringUtils.isBlank(
|
||||||
.isBlank(clusterAddRequest.getClusterType())) {
|
clusterAddRequest.getClusterType())) {
|
||||||
|
|
||||||
throw new SkyWalkerException("集群名字或者类型不能为空!");
|
throw new SkyWalkerException("集群名字或者类型不能为空!");
|
||||||
}
|
}
|
||||||
|
@ -76,9 +79,11 @@ public class ClusterAddProcessor implements Processor<ClusterAddRequest, Cluster
|
||||||
clusterDO.setClusterId(clusterId);
|
clusterDO.setClusterId(clusterId);
|
||||||
clusterDO.setClusterName(clusterAddRequest.getClusterName());
|
clusterDO.setClusterName(clusterAddRequest.getClusterName());
|
||||||
clusterDO.setClusterType(clusterAddRequest.getClusterType());
|
clusterDO.setClusterType(clusterAddRequest.getClusterType());
|
||||||
clusterDO.setConnectKeyList(JSONObject.toJSONString(clusterAddRequest.getConnectKeyList()));
|
clusterDO.setConnectKeyList(objectMapper.writeValueAsString(clusterAddRequest.getConnectKeyList()));
|
||||||
clusterDO.setUserName(clusterAddRequest.getUserName());
|
clusterDO.setUserName(clusterAddRequest.getUserName());
|
||||||
clusterDO.setPassword(clusterAddRequest.getPassword());
|
clusterDO.setPassword(clusterAddRequest.getPassword());
|
||||||
|
clusterDO.setNamespace(clusterAddRequest.getNamespace());
|
||||||
|
clusterDO.setClusterLevel(0);
|
||||||
clusterAccessService.insert(clusterDO);
|
clusterAccessService.insert(clusterDO);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,15 +14,16 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
import com.alibaba.nacossync.pojo.result.ClusterDeleteResult;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
|
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||||
import com.alibaba.nacossync.pojo.request.ClusterDeleteRequest;
|
import com.alibaba.nacossync.pojo.request.ClusterDeleteRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.result.ClusterDeleteResult;
|
||||||
import com.alibaba.nacossync.template.Processor;
|
import com.alibaba.nacossync.template.Processor;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
|
@ -31,12 +32,23 @@ import com.alibaba.nacossync.template.Processor;
|
||||||
@Service
|
@Service
|
||||||
public class ClusterDeleteProcessor implements Processor<ClusterDeleteRequest, ClusterDeleteResult> {
|
public class ClusterDeleteProcessor implements Processor<ClusterDeleteRequest, ClusterDeleteResult> {
|
||||||
|
|
||||||
@Autowired
|
private final ClusterAccessService clusterAccessService;
|
||||||
private ClusterAccessService clusterAccessService;
|
|
||||||
|
private final TaskAccessService taskAccessService;
|
||||||
|
|
||||||
|
public ClusterDeleteProcessor(ClusterAccessService clusterAccessService, TaskAccessService taskAccessService) {
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(ClusterDeleteRequest clusterDeleteRequest,
|
public void process(ClusterDeleteRequest clusterDeleteRequest, ClusterDeleteResult clusterDeleteResult,
|
||||||
ClusterDeleteResult clusterDeleteResult, Object... others) throws Exception {
|
Object... others) throws Exception {
|
||||||
|
int count = taskAccessService.countByDestClusterIdOrSourceClusterId(clusterDeleteRequest.getClusterId(),
|
||||||
|
clusterDeleteRequest.getClusterId());
|
||||||
|
if (count > 0) {
|
||||||
|
throw new SkyWalkerException(String.format("集群下有%d个任务,请先删除任务", count));
|
||||||
|
}
|
||||||
|
|
||||||
clusterAccessService.deleteByClusterId(clusterDeleteRequest.getClusterId());
|
clusterAccessService.deleteByClusterId(clusterDeleteRequest.getClusterId());
|
||||||
|
|
||||||
|
|
|
@ -14,44 +14,37 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
import com.alibaba.nacossync.pojo.result.ClusterDetailQueryResult;
|
|
||||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
import com.alibaba.nacossync.pojo.request.ClusterDetailQueryRequest;
|
import com.alibaba.nacossync.pojo.request.ClusterDetailQueryRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.result.ClusterDetailQueryResult;
|
||||||
import com.alibaba.nacossync.pojo.view.ClusterModel;
|
import com.alibaba.nacossync.pojo.view.ClusterModel;
|
||||||
import com.alibaba.nacossync.template.Processor;
|
import com.alibaba.nacossync.template.Processor;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: ClusterDetailQueryProcessor.java, v 0.1 2018-09-30 PM2:39 NacosSync Exp $$
|
* @version $Id: ClusterDetailQueryProcessor.java, v 0.1 2018-09-30 PM2:39 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
@Service
|
@Service
|
||||||
public class ClusterDetailQueryProcessor
|
public class ClusterDetailQueryProcessor implements Processor<ClusterDetailQueryRequest, ClusterDetailQueryResult> {
|
||||||
implements
|
|
||||||
Processor<ClusterDetailQueryRequest, ClusterDetailQueryResult> {
|
private final ClusterAccessService clusterAccessService;
|
||||||
@Autowired
|
|
||||||
private ClusterAccessService clusterAccessService;
|
public ClusterDetailQueryProcessor(ClusterAccessService clusterAccessService) {
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(ClusterDetailQueryRequest clusterDetailQueryRequest,
|
public void process(ClusterDetailQueryRequest clusterDetailQueryRequest,
|
||||||
ClusterDetailQueryResult clusterDetailQueryResult, Object... others)
|
ClusterDetailQueryResult clusterDetailQueryResult, Object... others) throws Exception {
|
||||||
throws Exception {
|
|
||||||
|
|
||||||
ClusterDO clusterDO = clusterAccessService.findByClusterId(clusterDetailQueryRequest
|
ClusterDO clusterDO = clusterAccessService.findByClusterId(clusterDetailQueryRequest.getClusterId());
|
||||||
.getClusterId());
|
|
||||||
|
|
||||||
ClusterModel clusterModel = new ClusterModel();
|
clusterDetailQueryResult.setClusterModel(ClusterModel.from(clusterDO));
|
||||||
clusterModel.setClusterId(clusterDO.getClusterId());
|
|
||||||
clusterModel.setConnectKeyList(clusterDO.getConnectKeyList());
|
|
||||||
clusterModel.setClusterType(clusterDO.getClusterType());
|
|
||||||
clusterModel.setClusterName(clusterDO.getClusterName());
|
|
||||||
|
|
||||||
clusterDetailQueryResult.setClusterModel(clusterModel);
|
|
||||||
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,38 +14,39 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import java.util.ArrayList;
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||||
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
|
import com.alibaba.nacossync.pojo.request.ClusterListQueryRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.result.ClusterListQueryResult;
|
||||||
|
import com.alibaba.nacossync.pojo.view.ClusterModel;
|
||||||
|
import com.alibaba.nacossync.template.Processor;
|
||||||
import org.apache.commons.lang3.StringUtils;
|
import org.apache.commons.lang3.StringUtils;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
import org.springframework.data.domain.Page;
|
import org.springframework.data.domain.Page;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.ClusterAccessService;
|
import java.util.List;
|
||||||
import com.alibaba.nacossync.pojo.result.ClusterListQueryResult;
|
|
||||||
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
|
||||||
import com.alibaba.nacossync.pojo.request.ClusterListQueryRequest;
|
|
||||||
import com.alibaba.nacossync.pojo.view.ClusterModel;
|
|
||||||
import com.alibaba.nacossync.template.Processor;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
* @version $Id: ClusterListQueryProcessor.java, v 0.1 2018-09-30 PM2:33 NacosSync Exp $$
|
* @version $Id: ClusterListQueryProcessor.java, v 0.1 2018-09-30 PM2:33 NacosSync Exp $$
|
||||||
*/
|
*/
|
||||||
@Service
|
@Service
|
||||||
public class ClusterListQueryProcessor implements
|
public class ClusterListQueryProcessor implements Processor<ClusterListQueryRequest, ClusterListQueryResult> {
|
||||||
Processor<ClusterListQueryRequest, ClusterListQueryResult> {
|
|
||||||
|
|
||||||
@Autowired
|
private final ClusterAccessService clusterAccessService;
|
||||||
private ClusterAccessService clusterAccessService;
|
|
||||||
|
public ClusterListQueryProcessor(ClusterAccessService clusterAccessService) {
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(ClusterListQueryRequest clusterListQueryRequest,
|
public void process(ClusterListQueryRequest clusterListQueryRequest, ClusterListQueryResult clusterListQueryResult,
|
||||||
ClusterListQueryResult clusterListQueryResult, Object... others) {
|
Object... others) {
|
||||||
|
|
||||||
Page<ClusterDO> clusterDOS;
|
Page<ClusterDO> clusterDOS;
|
||||||
|
|
||||||
|
@ -62,18 +63,7 @@ public class ClusterListQueryProcessor implements
|
||||||
clusterListQueryRequest.getPageSize());
|
clusterListQueryRequest.getPageSize());
|
||||||
|
|
||||||
}
|
}
|
||||||
|
List<ClusterModel> clusterModels = clusterDOS.stream().map(ClusterModel::from).toList();
|
||||||
List<ClusterModel> clusterModels = new ArrayList<>();
|
|
||||||
clusterDOS.forEach(clusterDO -> {
|
|
||||||
|
|
||||||
ClusterModel clusterModel = new ClusterModel();
|
|
||||||
clusterModel.setClusterId(clusterDO.getClusterId());
|
|
||||||
clusterModel.setClusterName(clusterDO.getClusterName());
|
|
||||||
clusterModel.setClusterType(clusterDO.getClusterType());
|
|
||||||
clusterModel.setConnectKeyList(clusterDO.getConnectKeyList());
|
|
||||||
clusterModels.add(clusterModel);
|
|
||||||
});
|
|
||||||
|
|
||||||
clusterListQueryResult.setClusterModels(clusterModels);
|
clusterListQueryResult.setClusterModels(clusterModels);
|
||||||
clusterListQueryResult.setTotalPage(clusterDOS.getTotalPages());
|
clusterListQueryResult.setTotalPage(clusterDOS.getTotalPages());
|
||||||
clusterListQueryResult.setCurrentSize(clusterModels.size());
|
clusterListQueryResult.setCurrentSize(clusterModels.size());
|
||||||
|
|
|
@ -0,0 +1,250 @@
|
||||||
|
/*
|
||||||
|
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||||
|
* contributor license agreements. See the NOTICE file distributed with
|
||||||
|
* this work for additional information regarding copyright ownership.
|
||||||
|
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||||
|
* (the "License"); you may not use this file except in compliance with
|
||||||
|
* the License. You may obtain a copy of the License at
|
||||||
|
*
|
||||||
|
* http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
*
|
||||||
|
* Unless required by applicable law or agreed to in writing, software
|
||||||
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
* See the License for the specific language governing permissions and
|
||||||
|
* limitations under the License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
|
import com.alibaba.nacos.api.exception.NacosException;
|
||||||
|
import com.alibaba.nacos.api.naming.CommonParams;
|
||||||
|
import com.alibaba.nacos.api.naming.NamingService;
|
||||||
|
import com.alibaba.nacos.client.naming.NacosNamingService;
|
||||||
|
import com.alibaba.nacos.client.naming.net.NamingProxy;
|
||||||
|
import com.alibaba.nacos.client.naming.utils.UtilAndComs;
|
||||||
|
import com.alibaba.nacos.common.utils.HttpMethod;
|
||||||
|
import com.alibaba.nacos.common.utils.JacksonUtils;
|
||||||
|
import com.alibaba.nacos.common.utils.StringUtils;
|
||||||
|
import com.alibaba.nacossync.constant.TaskStatusEnum;
|
||||||
|
import com.alibaba.nacossync.dao.ClusterAccessService;
|
||||||
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
|
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||||
|
import com.alibaba.nacossync.extension.SyncManagerService;
|
||||||
|
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
|
import com.alibaba.nacossync.pojo.model.ClusterDO;
|
||||||
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import com.alibaba.nacossync.pojo.request.TaskAddAllRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.result.TaskAddResult;
|
||||||
|
import com.alibaba.nacossync.template.Processor;
|
||||||
|
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
import org.springframework.util.ReflectionUtils;
|
||||||
|
|
||||||
|
import javax.annotation.Nullable;
|
||||||
|
import java.lang.reflect.Field;
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Objects;
|
||||||
|
|
||||||
|
import static com.alibaba.nacossync.constant.SkyWalkerConstants.GROUP_NAME_PARAM;
|
||||||
|
import static com.alibaba.nacossync.constant.SkyWalkerConstants.PAGE_NO;
|
||||||
|
import static com.alibaba.nacossync.constant.SkyWalkerConstants.PAGE_SIZE;
|
||||||
|
import static com.alibaba.nacossync.constant.SkyWalkerConstants.SERVICE_NAME_PARAM;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @author NacosSync
|
||||||
|
* @version $Id: TaskAddAllProcessor.java, v 0.1 2022-03-23 PM11:40 NacosSync Exp $$
|
||||||
|
*/
|
||||||
|
@Slf4j
|
||||||
|
@Service
|
||||||
|
public class TaskAddAllProcessor implements Processor<TaskAddAllRequest, TaskAddResult> {
|
||||||
|
|
||||||
|
private static final String CONSUMER_PREFIX = "consumers:";
|
||||||
|
|
||||||
|
private final NacosServerHolder nacosServerHolder;
|
||||||
|
|
||||||
|
private final SyncManagerService syncManagerService;
|
||||||
|
|
||||||
|
private final TaskAccessService taskAccessService;
|
||||||
|
|
||||||
|
private final ClusterAccessService clusterAccessService;
|
||||||
|
|
||||||
|
public TaskAddAllProcessor(NacosServerHolder nacosServerHolder, SyncManagerService syncManagerService,
|
||||||
|
TaskAccessService taskAccessService, ClusterAccessService clusterAccessService) {
|
||||||
|
this.nacosServerHolder = nacosServerHolder;
|
||||||
|
this.syncManagerService = syncManagerService;
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
this.clusterAccessService = clusterAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void process(TaskAddAllRequest addAllRequest, TaskAddResult taskAddResult, Object... others)
|
||||||
|
throws Exception {
|
||||||
|
|
||||||
|
ClusterDO destCluster = clusterAccessService.findByClusterId(addAllRequest.getDestClusterId());
|
||||||
|
|
||||||
|
ClusterDO sourceCluster = clusterAccessService.findByClusterId(addAllRequest.getSourceClusterId());
|
||||||
|
|
||||||
|
if (Objects.isNull(destCluster) || Objects.isNull(sourceCluster)) {
|
||||||
|
throw new SkyWalkerException("Please check if the source or target cluster exists.");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (Objects.isNull(syncManagerService.getSyncService(sourceCluster.getClusterId(), destCluster.getClusterId()))) {
|
||||||
|
throw new SkyWalkerException("current sync type not supported.");
|
||||||
|
}
|
||||||
|
// TODO 目前仅支持 Nacos 为源的同步类型,待完善更多类型支持。
|
||||||
|
final NamingService sourceNamingService = nacosServerHolder.get(sourceCluster.getClusterId());
|
||||||
|
if (sourceNamingService == null) {
|
||||||
|
throw new SkyWalkerException("only support sync type that the source of the Nacos.");
|
||||||
|
}
|
||||||
|
|
||||||
|
final EnhanceNamingService enhanceNamingService = new EnhanceNamingService(sourceNamingService);
|
||||||
|
final CatalogServiceResult catalogServiceResult = enhanceNamingService.catalogServices(null, null);
|
||||||
|
if (catalogServiceResult == null || catalogServiceResult.getCount() <= 0) {
|
||||||
|
throw new SkyWalkerException("sourceCluster data empty");
|
||||||
|
}
|
||||||
|
|
||||||
|
for (ServiceView serviceView : catalogServiceResult.getServiceList()) {
|
||||||
|
// exclude subscriber
|
||||||
|
if (addAllRequest.isExcludeConsumer() && serviceView.getName().startsWith(CONSUMER_PREFIX)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
TaskAddRequest taskAddRequest = new TaskAddRequest();
|
||||||
|
taskAddRequest.setSourceClusterId(sourceCluster.getClusterId());
|
||||||
|
taskAddRequest.setDestClusterId(destCluster.getClusterId());
|
||||||
|
taskAddRequest.setServiceName(serviceView.getName());
|
||||||
|
taskAddRequest.setGroupName(serviceView.getGroupName());
|
||||||
|
this.dealTask(addAllRequest, taskAddRequest);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void dealTask(TaskAddAllRequest addAllRequest, TaskAddRequest taskAddRequest) throws Exception {
|
||||||
|
|
||||||
|
String taskId = SkyWalkerUtil.generateTaskId(taskAddRequest);
|
||||||
|
TaskDO taskDO = taskAccessService.findByTaskId(taskId);
|
||||||
|
if (null == taskDO) {
|
||||||
|
taskDO = new TaskDO();
|
||||||
|
taskDO.setTaskId(taskId);
|
||||||
|
taskDO.setDestClusterId(addAllRequest.getDestClusterId());
|
||||||
|
taskDO.setSourceClusterId(addAllRequest.getSourceClusterId());
|
||||||
|
taskDO.setServiceName(taskAddRequest.getServiceName());
|
||||||
|
taskDO.setVersion(taskAddRequest.getVersion());
|
||||||
|
taskDO.setGroupName(taskAddRequest.getGroupName());
|
||||||
|
taskDO.setNameSpace(taskAddRequest.getNameSpace());
|
||||||
|
taskDO.setTaskStatus(TaskStatusEnum.SYNC.getCode());
|
||||||
|
taskDO.setWorkerIp(SkyWalkerUtil.getLocalIp());
|
||||||
|
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
||||||
|
|
||||||
|
} else {
|
||||||
|
taskDO.setTaskStatus(TaskStatusEnum.SYNC.getCode());
|
||||||
|
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
||||||
|
}
|
||||||
|
taskAccessService.addTask(taskDO);
|
||||||
|
}
|
||||||
|
|
||||||
|
static class EnhanceNamingService {
|
||||||
|
|
||||||
|
protected NamingService delegate;
|
||||||
|
|
||||||
|
protected NamingProxy serverProxy;
|
||||||
|
|
||||||
|
protected EnhanceNamingService(NamingService namingService) {
|
||||||
|
if (!(namingService instanceof NacosNamingService)) {
|
||||||
|
throw new IllegalArgumentException(
|
||||||
|
"namingService only support instance of com.alibaba.nacos.client.naming.NacosNamingService.");
|
||||||
|
}
|
||||||
|
this.delegate = namingService;
|
||||||
|
|
||||||
|
// serverProxy
|
||||||
|
final Field serverProxyField = ReflectionUtils.findField(NacosNamingService.class, "serverProxy");
|
||||||
|
assert serverProxyField != null;
|
||||||
|
ReflectionUtils.makeAccessible(serverProxyField);
|
||||||
|
this.serverProxy = (NamingProxy) ReflectionUtils.getField(serverProxyField, delegate);
|
||||||
|
}
|
||||||
|
|
||||||
|
public CatalogServiceResult catalogServices(@Nullable String serviceName, @Nullable String group)
|
||||||
|
throws NacosException {
|
||||||
|
int pageNo = 1; // start with 1
|
||||||
|
int pageSize = 100;
|
||||||
|
|
||||||
|
final CatalogServiceResult result = catalogServices(serviceName, group, pageNo, pageSize);
|
||||||
|
|
||||||
|
CatalogServiceResult tmpResult = result;
|
||||||
|
|
||||||
|
while (Objects.nonNull(tmpResult) && tmpResult.serviceList.size() >= pageSize) {
|
||||||
|
pageNo++;
|
||||||
|
tmpResult = catalogServices(serviceName, group, pageNo, pageSize);
|
||||||
|
|
||||||
|
if (tmpResult != null) {
|
||||||
|
result.serviceList.addAll(tmpResult.serviceList);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @see com.alibaba.nacos.client.naming.core.HostReactor#getServiceInfoDirectlyFromServer(String, String)
|
||||||
|
*/
|
||||||
|
public CatalogServiceResult catalogServices(@Nullable String serviceName, @Nullable String group, int pageNo,
|
||||||
|
int pageSize) throws NacosException {
|
||||||
|
|
||||||
|
// pageNo
|
||||||
|
// pageSize
|
||||||
|
// serviceNameParam
|
||||||
|
// groupNameParam
|
||||||
|
final Map<String, String> params = new HashMap<>(8);
|
||||||
|
params.put(CommonParams.NAMESPACE_ID, serverProxy.getNamespaceId());
|
||||||
|
params.put(SERVICE_NAME_PARAM, serviceName);
|
||||||
|
params.put(GROUP_NAME_PARAM, group);
|
||||||
|
params.put(PAGE_NO, String.valueOf(pageNo));
|
||||||
|
params.put(PAGE_SIZE, String.valueOf(pageSize));
|
||||||
|
|
||||||
|
final String result = this.serverProxy.reqApi(UtilAndComs.nacosUrlBase + "/catalog/services", params,
|
||||||
|
HttpMethod.GET);
|
||||||
|
if (StringUtils.isNotEmpty(result)) {
|
||||||
|
return JacksonUtils.toObj(result, CatalogServiceResult.class);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Copy from Nacos Server.
|
||||||
|
*/
|
||||||
|
@Data
|
||||||
|
static class ServiceView {
|
||||||
|
|
||||||
|
private String name;
|
||||||
|
|
||||||
|
private String groupName;
|
||||||
|
|
||||||
|
private int clusterCount;
|
||||||
|
|
||||||
|
private int ipCount;
|
||||||
|
|
||||||
|
private int healthyInstanceCount;
|
||||||
|
|
||||||
|
private String triggerFlag;
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
@Data
|
||||||
|
static class CatalogServiceResult {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* count,not equal serviceList.size .
|
||||||
|
*/
|
||||||
|
private int count;
|
||||||
|
|
||||||
|
private List<ServiceView> serviceList;
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
|
@ -16,19 +16,12 @@
|
||||||
*/
|
*/
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import java.util.ArrayList;
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
|
||||||
import com.alibaba.nacossync.pojo.request.TaskDeleteInBatchRequest;
|
import com.alibaba.nacossync.pojo.request.TaskDeleteInBatchRequest;
|
||||||
import com.alibaba.nacossync.pojo.result.BaseResult;
|
import com.alibaba.nacossync.pojo.result.BaseResult;
|
||||||
import com.alibaba.nacossync.template.Processor;
|
import com.alibaba.nacossync.template.Processor;
|
||||||
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -40,20 +33,15 @@ import lombok.extern.slf4j.Slf4j;
|
||||||
@Service
|
@Service
|
||||||
public class TaskDeleteInBatchProcessor implements Processor<TaskDeleteInBatchRequest, BaseResult> {
|
public class TaskDeleteInBatchProcessor implements Processor<TaskDeleteInBatchRequest, BaseResult> {
|
||||||
|
|
||||||
@Autowired
|
private final TaskAccessService taskAccessService;
|
||||||
private TaskAccessService taskAccessService;
|
|
||||||
|
public TaskDeleteInBatchProcessor(TaskAccessService taskAccessService) {
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(TaskDeleteInBatchRequest taskBatchDeleteRequest, BaseResult baseResult,
|
public void process(TaskDeleteInBatchRequest taskBatchDeleteRequest, BaseResult baseResult,
|
||||||
Object... others) {
|
Object... others) {
|
||||||
//
|
|
||||||
// String[] taskIds= taskBatchDeleteRequest.getTaskIds();
|
|
||||||
// List<TaskDO> taskDOs = new ArrayList<TaskDO>();
|
|
||||||
// for (String taskId : taskIds) {
|
|
||||||
// TaskDO taskDO = new TaskDO();
|
|
||||||
// taskDO.setTaskId(taskId);
|
|
||||||
// taskDOs.add(taskDO);
|
|
||||||
// }
|
|
||||||
taskAccessService.deleteTaskInBatch(taskBatchDeleteRequest.getTaskIds());
|
taskAccessService.deleteTaskInBatch(taskBatchDeleteRequest.getTaskIds());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,9 +14,12 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
|
import com.alibaba.nacossync.constant.SkyWalkerConstants;
|
||||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
|
import com.alibaba.nacossync.event.DeleteAllSubTaskEvent;
|
||||||
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.pojo.request.TaskDeleteRequest;
|
import com.alibaba.nacossync.pojo.request.TaskDeleteRequest;
|
||||||
|
@ -24,7 +27,6 @@ import com.alibaba.nacossync.pojo.result.BaseResult;
|
||||||
import com.alibaba.nacossync.template.Processor;
|
import com.alibaba.nacossync.template.Processor;
|
||||||
import com.google.common.eventbus.EventBus;
|
import com.google.common.eventbus.EventBus;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -35,17 +37,27 @@ import org.springframework.stereotype.Service;
|
||||||
@Service
|
@Service
|
||||||
public class TaskDeleteProcessor implements Processor<TaskDeleteRequest, BaseResult> {
|
public class TaskDeleteProcessor implements Processor<TaskDeleteRequest, BaseResult> {
|
||||||
|
|
||||||
@Autowired
|
private final TaskAccessService taskAccessService;
|
||||||
private TaskAccessService taskAccessService;
|
|
||||||
@Autowired
|
private final EventBus eventBus;
|
||||||
private EventBus eventBus;
|
|
||||||
|
public TaskDeleteProcessor(TaskAccessService taskAccessService, EventBus eventBus) {
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
this.eventBus = eventBus;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(TaskDeleteRequest taskDeleteRequest, BaseResult baseResult,
|
public void process(TaskDeleteRequest taskDeleteRequest, BaseResult baseResult, Object... others) {
|
||||||
Object... others) {
|
|
||||||
TaskDO taskDO = taskAccessService.findByTaskId(taskDeleteRequest.getTaskId());
|
TaskDO taskDO = taskAccessService.findByTaskId(taskDeleteRequest.getTaskId());
|
||||||
eventBus.post(new DeleteTaskEvent(taskDO));
|
// delete all sub task when ServiceName is all
|
||||||
log.info("删除同步任务数据之前,发出一个同步事件:" + taskDO);
|
if (SkyWalkerConstants.NACOS_ALL_SERVICE_NAME.equalsIgnoreCase(taskDO.getServiceName())) {
|
||||||
|
eventBus.post(new DeleteAllSubTaskEvent(taskDO));
|
||||||
|
} else {
|
||||||
|
eventBus.post(new DeleteTaskEvent(taskDO));
|
||||||
|
}
|
||||||
|
log.info("删除同步任务数据之前,发出一个同步事件:{}", taskDO);
|
||||||
taskAccessService.deleteTaskById(taskDeleteRequest.getTaskId());
|
taskAccessService.deleteTaskById(taskDeleteRequest.getTaskId());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,17 +14,17 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||||
import com.alibaba.nacossync.pojo.result.TaskDetailQueryResult;
|
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.pojo.request.TaskDetailQueryRequest;
|
import com.alibaba.nacossync.pojo.request.TaskDetailQueryRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.result.TaskDetailQueryResult;
|
||||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
import com.alibaba.nacossync.pojo.view.TaskModel;
|
||||||
import com.alibaba.nacossync.template.Processor;
|
import com.alibaba.nacossync.template.Processor;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -35,12 +35,15 @@ import org.springframework.stereotype.Service;
|
||||||
@Service
|
@Service
|
||||||
public class TaskDetailProcessor implements Processor<TaskDetailQueryRequest, TaskDetailQueryResult> {
|
public class TaskDetailProcessor implements Processor<TaskDetailQueryRequest, TaskDetailQueryResult> {
|
||||||
|
|
||||||
@Autowired
|
private final TaskAccessService taskAccessService;
|
||||||
private TaskAccessService taskAccessService;
|
|
||||||
|
public TaskDetailProcessor(TaskAccessService taskAccessService) {
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(TaskDetailQueryRequest taskDetailQueryRequest, TaskDetailQueryResult taskDetailQueryResult, Object... others)
|
public void process(TaskDetailQueryRequest taskDetailQueryRequest, TaskDetailQueryResult taskDetailQueryResult,
|
||||||
throws Exception {
|
Object... others) throws Exception {
|
||||||
|
|
||||||
TaskDO taskDO = taskAccessService.findByTaskId(taskDetailQueryRequest.getTaskId());
|
TaskDO taskDO = taskAccessService.findByTaskId(taskDetailQueryRequest.getTaskId());
|
||||||
|
|
||||||
|
@ -48,15 +51,6 @@ public class TaskDetailProcessor implements Processor<TaskDetailQueryRequest, Ta
|
||||||
throw new SkyWalkerException("taskDo is null,taskId :" + taskDetailQueryRequest.getTaskId());
|
throw new SkyWalkerException("taskDo is null,taskId :" + taskDetailQueryRequest.getTaskId());
|
||||||
}
|
}
|
||||||
|
|
||||||
TaskModel taskModel = new TaskModel();
|
taskDetailQueryResult.setTaskModel(TaskModel.from(taskDO));
|
||||||
|
|
||||||
taskModel.setDestClusterId(taskDO.getDestClusterId());
|
|
||||||
taskModel.setGroupName(taskDO.getGroupName());
|
|
||||||
taskModel.setServiceName(taskDO.getServiceName());
|
|
||||||
taskModel.setSourceClusterId(taskDO.getSourceClusterId());
|
|
||||||
taskModel.setTaskStatus(taskDO.getTaskStatus());
|
|
||||||
taskModel.setTaskId(taskDO.getTaskId());
|
|
||||||
|
|
||||||
taskDetailQueryResult.setTaskModel(taskModel);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -14,25 +14,23 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import java.util.ArrayList;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.pojo.QueryCondition;
|
import com.alibaba.nacossync.pojo.QueryCondition;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import com.alibaba.nacossync.pojo.request.TaskListQueryRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.result.TaskListQueryResult;
|
||||||
|
import com.alibaba.nacossync.pojo.view.TaskModel;
|
||||||
|
import com.alibaba.nacossync.template.Processor;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
|
||||||
import org.apache.commons.lang3.StringUtils;
|
import org.apache.commons.lang3.StringUtils;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
import org.springframework.data.domain.Page;
|
import org.springframework.data.domain.Page;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
import java.util.List;
|
||||||
import com.alibaba.nacossync.pojo.result.TaskListQueryResult;
|
|
||||||
import com.alibaba.nacossync.pojo.request.TaskListQueryRequest;
|
|
||||||
import com.alibaba.nacossync.pojo.view.TaskModel;
|
|
||||||
import com.alibaba.nacossync.template.Processor;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
|
@ -42,12 +40,15 @@ import com.alibaba.nacossync.template.Processor;
|
||||||
@Slf4j
|
@Slf4j
|
||||||
public class TaskListQueryProcessor implements Processor<TaskListQueryRequest, TaskListQueryResult> {
|
public class TaskListQueryProcessor implements Processor<TaskListQueryRequest, TaskListQueryResult> {
|
||||||
|
|
||||||
@Autowired
|
private final TaskAccessService taskAccessService;
|
||||||
private TaskAccessService taskAccessService;
|
|
||||||
|
public TaskListQueryProcessor(TaskAccessService taskAccessService) {
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(TaskListQueryRequest taskListQueryRequest,
|
public void process(TaskListQueryRequest taskListQueryRequest, TaskListQueryResult taskListQueryResult,
|
||||||
TaskListQueryResult taskListQueryResult, Object... others) {
|
Object... others) {
|
||||||
|
|
||||||
Page<TaskDO> taskDOPage;
|
Page<TaskDO> taskDOPage;
|
||||||
|
|
||||||
|
@ -64,18 +65,7 @@ public class TaskListQueryProcessor implements Processor<TaskListQueryRequest, T
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
List<TaskModel> taskList = new ArrayList<>();
|
List<TaskModel> taskList = taskDOPage.stream().map(TaskModel::from).toList();
|
||||||
|
|
||||||
taskDOPage.forEach(taskDO -> {
|
|
||||||
TaskModel taskModel = new TaskModel();
|
|
||||||
taskModel.setTaskId(taskDO.getTaskId());
|
|
||||||
taskModel.setDestClusterId(taskDO.getDestClusterId());
|
|
||||||
taskModel.setSourceClusterId(taskDO.getSourceClusterId());
|
|
||||||
taskModel.setServiceName(taskDO.getServiceName());
|
|
||||||
taskModel.setGroupName(taskDO.getGroupName());
|
|
||||||
taskModel.setTaskStatus(taskDO.getTaskStatus());
|
|
||||||
taskList.add(taskModel);
|
|
||||||
});
|
|
||||||
|
|
||||||
taskListQueryResult.setTaskModels(taskList);
|
taskListQueryResult.setTaskModels(taskList);
|
||||||
taskListQueryResult.setTotalPage(taskDOPage.getTotalPages());
|
taskListQueryResult.setTotalPage(taskDOPage.getTotalPages());
|
||||||
|
|
|
@ -14,21 +14,19 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.template.processor;
|
package com.alibaba.nacossync.template.processor;
|
||||||
|
|
||||||
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.TaskStatusEnum;
|
import com.alibaba.nacossync.constant.TaskStatusEnum;
|
||||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
import com.alibaba.nacossync.exception.SkyWalkerException;
|
import com.alibaba.nacossync.exception.SkyWalkerException;
|
||||||
import com.alibaba.nacossync.pojo.result.BaseResult;
|
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.pojo.request.TaskUpdateRequest;
|
import com.alibaba.nacossync.pojo.request.TaskUpdateRequest;
|
||||||
|
import com.alibaba.nacossync.pojo.result.BaseResult;
|
||||||
import com.alibaba.nacossync.template.Processor;
|
import com.alibaba.nacossync.template.Processor;
|
||||||
|
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
|
@ -37,29 +35,33 @@ import com.alibaba.nacossync.template.Processor;
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@Service
|
@Service
|
||||||
public class TaskUpdateProcessor implements Processor<TaskUpdateRequest, BaseResult> {
|
public class TaskUpdateProcessor implements Processor<TaskUpdateRequest, BaseResult> {
|
||||||
@Autowired
|
|
||||||
private TaskAccessService taskAccessService;
|
private final TaskAccessService taskAccessService;
|
||||||
|
|
||||||
|
public TaskUpdateProcessor(TaskAccessService taskAccessService) {
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void process(TaskUpdateRequest taskUpdateRequest, BaseResult baseResult,
|
public void process(TaskUpdateRequest taskUpdateRequest, BaseResult baseResult, Object... others) throws Exception {
|
||||||
Object... others) throws Exception {
|
|
||||||
|
|
||||||
TaskDO taskDO = taskAccessService.findByTaskId(taskUpdateRequest.getTaskId());
|
TaskDO taskDO = taskAccessService.findByTaskId(taskUpdateRequest.getTaskId());
|
||||||
|
|
||||||
if (!TaskStatusEnum.contains(taskUpdateRequest.getTaskStatus())) {
|
if (!TaskStatusEnum.contains(taskUpdateRequest.getTaskStatus())) {
|
||||||
throw new SkyWalkerException(
|
throw new SkyWalkerException(
|
||||||
"taskUpdateRequest.getTaskStatus() is not exist , value is :"
|
"taskUpdateRequest.getTaskStatus() is not exist , value is :" + taskUpdateRequest.getTaskStatus());
|
||||||
+ taskUpdateRequest.getTaskStatus());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (null == taskDO) {
|
if (null == taskDO) {
|
||||||
throw new SkyWalkerException("taskDo is null ,taskId is :"
|
throw new SkyWalkerException("taskDo is null ,taskId is :" + taskUpdateRequest.getTaskId());
|
||||||
+ taskUpdateRequest.getTaskId());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
taskDO.setTaskStatus(taskUpdateRequest.getTaskStatus());
|
taskDO.setTaskStatus(taskUpdateRequest.getTaskStatus());
|
||||||
|
|
||||||
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
taskDO.setOperationId(SkyWalkerUtil.generateOperationId());
|
||||||
|
|
||||||
taskAccessService.addTask(taskDO);
|
taskAccessService.addTask(taskDO);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -0,0 +1,252 @@
|
||||||
|
/*
|
||||||
|
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||||
|
* contributor license agreements. See the NOTICE file distributed with
|
||||||
|
* this work for additional information regarding copyright ownership.
|
||||||
|
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||||
|
* (the "License"); you may not use this file except in compliance with
|
||||||
|
* the License. You may obtain a copy of the License at
|
||||||
|
*
|
||||||
|
* http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
*
|
||||||
|
* Unless required by applicable law or agreed to in writing, software
|
||||||
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
* See the License for the specific language governing permissions and
|
||||||
|
* limitations under the License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package com.alibaba.nacossync.timer;
|
||||||
|
|
||||||
|
import com.alibaba.nacos.api.naming.NamingService;
|
||||||
|
import com.alibaba.nacos.api.naming.pojo.ListView;
|
||||||
|
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
|
||||||
|
import com.alibaba.nacossync.constant.MetricsStatisticsType;
|
||||||
|
import com.alibaba.nacossync.constant.TaskStatusEnum;
|
||||||
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
|
import com.alibaba.nacossync.event.DeleteAllSubTaskEvent;
|
||||||
|
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||||
|
import com.alibaba.nacossync.event.SyncTaskEvent;
|
||||||
|
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import com.alibaba.nacossync.util.BatchTaskExecutor;
|
||||||
|
import com.alibaba.nacossync.util.SkyWalkerUtil;
|
||||||
|
import com.google.common.eventbus.EventBus;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.beans.BeanUtils;
|
||||||
|
|
||||||
|
import java.util.Collections;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* when the database task service name is empty, check all the services in the cluster and create a synchronization
|
||||||
|
* task.
|
||||||
|
*/
|
||||||
|
@Slf4j
|
||||||
|
public class CheckRunningStatusAllNacosThread implements Runnable {
|
||||||
|
|
||||||
|
private final Map<String, Set<String>> subTaskService = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
|
private final MetricsManager metricsManager;
|
||||||
|
|
||||||
|
private final TaskAccessService taskAccessService;
|
||||||
|
|
||||||
|
private final NacosServerHolder nacosServerHolder;
|
||||||
|
|
||||||
|
private final EventBus eventBus;
|
||||||
|
|
||||||
|
|
||||||
|
public CheckRunningStatusAllNacosThread(MetricsManager metricsManager, TaskAccessService taskAccessService,
|
||||||
|
NacosServerHolder nacosServerHolder, EventBus eventBus) {
|
||||||
|
this.metricsManager = metricsManager;
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
this.nacosServerHolder = nacosServerHolder;
|
||||||
|
this.eventBus = eventBus;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Synchronize data based on the ns level.
|
||||||
|
*/
|
||||||
|
@Override
|
||||||
|
public void run() {
|
||||||
|
|
||||||
|
try {
|
||||||
|
List<TaskDO> tasks = taskAccessService.findAllByServiceNameEqualAll();
|
||||||
|
if (CollectionUtils.isEmpty(tasks)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get the set of all task IDs
|
||||||
|
Set<String> taskIdSet = tasks.stream().map(TaskDO::getTaskId).collect(Collectors.toSet());
|
||||||
|
|
||||||
|
// Filter and handle sub-tasks that need to be deleted. This handles the case where tasks have been deleted
|
||||||
|
// but sub-tasks still exist.
|
||||||
|
subTaskService.entrySet().stream()
|
||||||
|
.filter(entry -> shouldDeleteSubTasks(entry, taskIdSet))
|
||||||
|
.forEach(entry -> postDeleteAllSubTaskEvent(entry.getKey()));
|
||||||
|
|
||||||
|
// Handle regular tasks
|
||||||
|
tasks.forEach(this::processTask);
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.warn("CheckRunningStatusThread Exception", e);
|
||||||
|
metricsManager.recordError(MetricsStatisticsType.DISPATCHER_TASK);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Listens for the event of deleting all sub-tasks and handles the delete operation.
|
||||||
|
*
|
||||||
|
* @param deleteAllSubTaskEvent The event object containing the task information to be deleted.
|
||||||
|
*/
|
||||||
|
public void listenerDeleteAllTaskEvent(DeleteAllSubTaskEvent deleteAllSubTaskEvent) {
|
||||||
|
// Retrieve the task object
|
||||||
|
TaskDO task = deleteAllSubTaskEvent.getTaskDO();
|
||||||
|
|
||||||
|
// Retrieve the task ID
|
||||||
|
String taskId = task.getTaskId();
|
||||||
|
|
||||||
|
// Remove the set of service names corresponding to the task ID from subTaskService
|
||||||
|
Set<String> serviceNameSet = subTaskService.remove(taskId);
|
||||||
|
|
||||||
|
// If the set of service names is empty, return immediately
|
||||||
|
if (CollectionUtils.isEmpty(serviceNameSet)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build the list of sub-tasks pending removal
|
||||||
|
List<TaskDO> servicesPendingRemoval = serviceNameSet.stream()
|
||||||
|
.map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||||
|
.toList();
|
||||||
|
|
||||||
|
// Handle the removal of the pending sub-tasks
|
||||||
|
handleRemoval(servicesPendingRemoval, serviceNameSet);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
private boolean shouldDeleteSubTasks(Map.Entry<String, Set<String>> entry, Set<String> taskIdSet) {
|
||||||
|
return !taskIdSet.contains(entry.getKey()) && !entry.getValue().isEmpty();
|
||||||
|
}
|
||||||
|
|
||||||
|
private void postDeleteAllSubTaskEvent(String taskId) {
|
||||||
|
TaskDO taskDO = new TaskDO();
|
||||||
|
taskDO.setTaskId(taskId);
|
||||||
|
eventBus.post(new DeleteAllSubTaskEvent(taskDO));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Processes the given task by determining the services that need to be inserted and removed,
|
||||||
|
* and performs the corresponding operations.
|
||||||
|
*
|
||||||
|
* @param task The task object to be processed.
|
||||||
|
*/
|
||||||
|
private void processTask(TaskDO task) {
|
||||||
|
// Retrieve the set of services for the task, creating a new set if it does not exist
|
||||||
|
Set<String> serviceSet = subTaskService.computeIfAbsent(task.getTaskId(), k -> ConcurrentHashMap.newKeySet());
|
||||||
|
|
||||||
|
// Get the list of all service names associated with the task
|
||||||
|
List<String> serviceNameList = getAllServiceName(task);
|
||||||
|
|
||||||
|
// Determine the services that need to be inserted (those not in the current service set)
|
||||||
|
List<TaskDO> servicesPendingInsertion = serviceNameList.stream()
|
||||||
|
.filter(serviceName -> !serviceSet.contains(serviceName))
|
||||||
|
.map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||||
|
.toList();
|
||||||
|
|
||||||
|
// Determine the services that need to be removed (those in the current service set but not in the service name list)
|
||||||
|
List<TaskDO> servicesPendingRemoval = serviceSet.stream()
|
||||||
|
.filter(serviceName -> !serviceNameList.contains(serviceName))
|
||||||
|
.map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||||
|
.toList();
|
||||||
|
|
||||||
|
|
||||||
|
// If all lists are empty, there is nothing to process
|
||||||
|
if (CollectionUtils.isEmpty(serviceNameList) && CollectionUtils.isEmpty(servicesPendingInsertion)
|
||||||
|
&& CollectionUtils.isEmpty(servicesPendingRemoval)) {
|
||||||
|
log.debug("No service found for task: {}", task.getTaskId());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// If the task status is SYNC, handle the insertion of services
|
||||||
|
if (TaskStatusEnum.SYNC.getCode().equals(task.getTaskStatus())) {
|
||||||
|
handleInsertion(servicesPendingInsertion, serviceSet);
|
||||||
|
}
|
||||||
|
// Handle the removal of services
|
||||||
|
handleRemoval(servicesPendingRemoval, serviceSet);
|
||||||
|
|
||||||
|
if (TaskStatusEnum.DELETE.getCode().equals(task.getTaskStatus())) {
|
||||||
|
List<TaskDO> allSubTasks = serviceNameList.stream().map(serviceName -> buildSubTaskDO(task, serviceName))
|
||||||
|
.collect(Collectors.toList());
|
||||||
|
handleRemoval(allSubTasks, serviceSet);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Handles the insertion of services.
|
||||||
|
*
|
||||||
|
* @param servicesPendingInsertion The list of services to be inserted.
|
||||||
|
* @param serviceSet The set of services.
|
||||||
|
*/
|
||||||
|
private void handleInsertion(List<TaskDO> servicesPendingInsertion, Set<String> serviceSet) {
|
||||||
|
BatchTaskExecutor.batchOperation(servicesPendingInsertion, t -> {
|
||||||
|
eventBus.post(new SyncTaskEvent(t));
|
||||||
|
serviceSet.add(t.getServiceName());
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handles the removal of services.
|
||||||
|
*
|
||||||
|
* @param servicesPendingRemoval The list of services to be removed.
|
||||||
|
* @param serviceSet The set of services.
|
||||||
|
*/
|
||||||
|
private void handleRemoval(List<TaskDO> servicesPendingRemoval, Set<String> serviceSet) {
|
||||||
|
BatchTaskExecutor.batchOperation(servicesPendingRemoval, t -> {
|
||||||
|
eventBus.post(new DeleteTaskEvent(t));
|
||||||
|
serviceSet.remove(t.getServiceName());
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Builds a sub-task object for the given task and service name.
|
||||||
|
*
|
||||||
|
* @param serviceName The service name.
|
||||||
|
* @return The constructed sub-task object.
|
||||||
|
*/
|
||||||
|
private static TaskDO buildSubTaskDO(TaskDO taskDO, String serviceName) {
|
||||||
|
TaskDO task = new TaskDO();
|
||||||
|
|
||||||
|
BeanUtils.copyProperties(taskDO, task);
|
||||||
|
task.setTaskId(SkyWalkerUtil.generateTaskId(serviceName, taskDO.getGroupName(), taskDO.getSourceClusterId(),
|
||||||
|
taskDO.getDestClusterId()));
|
||||||
|
task.setServiceName(serviceName);
|
||||||
|
return task;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Retrieves all service names associated with the given task.
|
||||||
|
*
|
||||||
|
* @return A list of service names.
|
||||||
|
*/
|
||||||
|
private List<String> getAllServiceName(TaskDO taskDO) {
|
||||||
|
NamingService namingService = nacosServerHolder.get(taskDO.getSourceClusterId());
|
||||||
|
if (namingService == null) {
|
||||||
|
log.warn("naming service is null or not found, clusterId:{}", taskDO.getSourceClusterId());
|
||||||
|
return Collections.emptyList();
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
ListView<String> servicesOfServer = namingService.getServicesOfServer(0, Integer.MAX_VALUE,
|
||||||
|
taskDO.getGroupName());
|
||||||
|
return servicesOfServer.getData();
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("query service list failure", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
return Collections.emptyList();
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
}
|
|
@ -14,24 +14,23 @@
|
||||||
* See the License for the specific language governing permissions and
|
* See the License for the specific language governing permissions and
|
||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
package com.alibaba.nacossync.timer;
|
package com.alibaba.nacossync.timer;
|
||||||
|
|
||||||
import java.util.HashSet;
|
|
||||||
import java.util.Map;
|
|
||||||
import java.util.Set;
|
|
||||||
import java.util.concurrent.ScheduledExecutorService;
|
|
||||||
import java.util.concurrent.TimeUnit;
|
|
||||||
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.boot.CommandLineRunner;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
import com.alibaba.nacossync.cache.SkyWalkerCacheServices;
|
||||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
import com.alibaba.nacossync.pojo.FinishedTask;
|
import com.alibaba.nacossync.pojo.FinishedTask;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.boot.CommandLineRunner;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.concurrent.ScheduledExecutorService;
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
import java.util.stream.StreamSupport;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* @author NacosSync
|
||||||
|
@ -41,20 +40,29 @@ import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
@Service
|
@Service
|
||||||
public class CleanExceedOperationIdTimer implements CommandLineRunner {
|
public class CleanExceedOperationIdTimer implements CommandLineRunner {
|
||||||
|
|
||||||
@Autowired
|
private static final long INITIAL_DELAY = 0;
|
||||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
|
||||||
|
|
||||||
@Autowired
|
private static final long PERIOD = 12;
|
||||||
private TaskAccessService taskAccessService;
|
|
||||||
|
|
||||||
@Autowired
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
private ScheduledExecutorService scheduledExecutorService;
|
|
||||||
|
private final TaskAccessService taskAccessService;
|
||||||
|
|
||||||
|
private final ScheduledExecutorService scheduledExecutorService;
|
||||||
|
|
||||||
|
public CleanExceedOperationIdTimer(SkyWalkerCacheServices skyWalkerCacheServices,
|
||||||
|
TaskAccessService taskAccessService, ScheduledExecutorService scheduledExecutorService) {
|
||||||
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
this.scheduledExecutorService = scheduledExecutorService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void run(String... args) {
|
public void run(String... args) {
|
||||||
/** Clean up the OperationId cache once every 12 hours */
|
/** Clean up the OperationId cache once every 12 hours */
|
||||||
scheduledExecutorService.scheduleWithFixedDelay(new CleanExceedOperationIdThread(), 0, 12,
|
scheduledExecutorService.scheduleWithFixedDelay(new CleanExceedOperationIdThread(), INITIAL_DELAY, PERIOD,
|
||||||
TimeUnit.HOURS);
|
TimeUnit.HOURS);
|
||||||
|
log.info("CleanExceedOperationIdTimer has started successfully");
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -65,20 +73,9 @@ public class CleanExceedOperationIdTimer implements CommandLineRunner {
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
|
||||||
Map<String, FinishedTask> finishedTaskMap = skyWalkerCacheServices
|
Map<String, FinishedTask> finishedTaskMap = skyWalkerCacheServices.getFinishedTaskMap();
|
||||||
.getFinishedTaskMap();
|
Set<String> operationIds = getDbOperations(taskAccessService.findAll());
|
||||||
|
finishedTaskMap.keySet().removeIf(operationId -> !operationIds.contains(operationId));
|
||||||
Iterable<TaskDO> taskDOS = taskAccessService.findAll();
|
|
||||||
|
|
||||||
Set<String> operationIds = getDbOperations(taskDOS);
|
|
||||||
|
|
||||||
for (String operationId : finishedTaskMap.keySet()) {
|
|
||||||
|
|
||||||
if (!operationIds.contains(operationId)) {
|
|
||||||
|
|
||||||
finishedTaskMap.remove(operationId);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.warn("CleanExceedOperationIdThread Exception", e);
|
log.warn("CleanExceedOperationIdThread Exception", e);
|
||||||
|
@ -87,10 +84,8 @@ public class CleanExceedOperationIdTimer implements CommandLineRunner {
|
||||||
}
|
}
|
||||||
|
|
||||||
private Set<String> getDbOperations(Iterable<TaskDO> taskDOS) {
|
private Set<String> getDbOperations(Iterable<TaskDO> taskDOS) {
|
||||||
Set<String> operationIds = new HashSet<>();
|
return StreamSupport.stream(taskDOS.spliterator(), false).map(TaskDO::getOperationId)
|
||||||
|
.collect(Collectors.toSet());
|
||||||
taskDOS.forEach(taskDO -> operationIds.add(taskDO.getOperationId()));
|
|
||||||
return operationIds;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -22,14 +22,15 @@ import com.alibaba.nacossync.constant.TaskStatusEnum;
|
||||||
import com.alibaba.nacossync.dao.TaskAccessService;
|
import com.alibaba.nacossync.dao.TaskAccessService;
|
||||||
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
import com.alibaba.nacossync.event.DeleteTaskEvent;
|
||||||
import com.alibaba.nacossync.event.SyncTaskEvent;
|
import com.alibaba.nacossync.event.SyncTaskEvent;
|
||||||
|
import com.alibaba.nacossync.extension.holder.NacosServerHolder;
|
||||||
import com.alibaba.nacossync.monitor.MetricsManager;
|
import com.alibaba.nacossync.monitor.MetricsManager;
|
||||||
import com.alibaba.nacossync.pojo.model.TaskDO;
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.google.common.eventbus.EventBus;
|
import com.google.common.eventbus.EventBus;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.boot.CommandLineRunner;
|
import org.springframework.boot.CommandLineRunner;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
import java.util.concurrent.ScheduledExecutorService;
|
import java.util.concurrent.ScheduledExecutorService;
|
||||||
import java.util.concurrent.TimeUnit;
|
import java.util.concurrent.TimeUnit;
|
||||||
|
|
||||||
|
@ -40,27 +41,44 @@ import java.util.concurrent.TimeUnit;
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@Service
|
@Service
|
||||||
public class QuerySyncTaskTimer implements CommandLineRunner {
|
public class QuerySyncTaskTimer implements CommandLineRunner {
|
||||||
@Autowired
|
|
||||||
private MetricsManager metricsManager;
|
|
||||||
|
|
||||||
@Autowired
|
private static final int INITIAL_DELAY = 0;
|
||||||
private SkyWalkerCacheServices skyWalkerCacheServices;
|
|
||||||
|
|
||||||
@Autowired
|
private static final int DELAY = 3000;
|
||||||
private TaskAccessService taskAccessService;
|
|
||||||
|
|
||||||
@Autowired
|
private final MetricsManager metricsManager;
|
||||||
private EventBus eventBus;
|
|
||||||
|
|
||||||
@Autowired
|
private final SkyWalkerCacheServices skyWalkerCacheServices;
|
||||||
private ScheduledExecutorService scheduledExecutorService;
|
|
||||||
|
private final TaskAccessService taskAccessService;
|
||||||
|
|
||||||
|
private final EventBus eventBus;
|
||||||
|
|
||||||
|
private final ScheduledExecutorService scheduledExecutorService;
|
||||||
|
|
||||||
|
private final NacosServerHolder nacosServerHolder;
|
||||||
|
|
||||||
|
public QuerySyncTaskTimer(MetricsManager metricsManager, SkyWalkerCacheServices skyWalkerCacheServices,
|
||||||
|
TaskAccessService taskAccessService, EventBus eventBus, ScheduledExecutorService scheduledExecutorService,
|
||||||
|
NacosServerHolder nacosServerHolder) {
|
||||||
|
this.metricsManager = metricsManager;
|
||||||
|
this.skyWalkerCacheServices = skyWalkerCacheServices;
|
||||||
|
this.taskAccessService = taskAccessService;
|
||||||
|
this.eventBus = eventBus;
|
||||||
|
this.scheduledExecutorService = scheduledExecutorService;
|
||||||
|
this.nacosServerHolder = nacosServerHolder;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void run(String... args) {
|
public void run(String... args) {
|
||||||
/** Fetch the task list from the database every 3 seconds */
|
/** Fetch the task list from the database every 3 seconds */
|
||||||
scheduledExecutorService.scheduleWithFixedDelay(new CheckRunningStatusThread(), 0, 3000,
|
scheduledExecutorService.scheduleWithFixedDelay(new CheckRunningStatusThread(), INITIAL_DELAY, DELAY,
|
||||||
TimeUnit.MILLISECONDS);
|
TimeUnit.MILLISECONDS);
|
||||||
|
|
||||||
|
scheduledExecutorService.scheduleWithFixedDelay(new CheckRunningStatusAllNacosThread(metricsManager,
|
||||||
|
taskAccessService, nacosServerHolder, eventBus), INITIAL_DELAY, DELAY,
|
||||||
|
TimeUnit.MILLISECONDS);
|
||||||
|
log.info("QuerySyncTaskTimer has started successfully");
|
||||||
}
|
}
|
||||||
|
|
||||||
private class CheckRunningStatusThread implements Runnable {
|
private class CheckRunningStatusThread implements Runnable {
|
||||||
|
@ -68,10 +86,10 @@ public class QuerySyncTaskTimer implements CommandLineRunner {
|
||||||
@Override
|
@Override
|
||||||
public void run() {
|
public void run() {
|
||||||
|
|
||||||
Long start = System.currentTimeMillis();
|
long start = System.currentTimeMillis();
|
||||||
try {
|
try {
|
||||||
|
|
||||||
Iterable<TaskDO> taskDOS = taskAccessService.findAll();
|
List<TaskDO> taskDOS = taskAccessService.findAllByServiceNameNotEqualAll();
|
||||||
|
|
||||||
taskDOS.forEach(taskDO -> {
|
taskDOS.forEach(taskDO -> {
|
||||||
|
|
||||||
|
@ -83,13 +101,13 @@ public class QuerySyncTaskTimer implements CommandLineRunner {
|
||||||
if (TaskStatusEnum.SYNC.getCode().equals(taskDO.getTaskStatus())) {
|
if (TaskStatusEnum.SYNC.getCode().equals(taskDO.getTaskStatus())) {
|
||||||
|
|
||||||
eventBus.post(new SyncTaskEvent(taskDO));
|
eventBus.post(new SyncTaskEvent(taskDO));
|
||||||
log.info("从数据库中查询到一个同步任务,发出一个同步事件:" + taskDO);
|
log.info("从数据库中查询到一个同步任务,发出一个同步事件:{}", taskDO);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (TaskStatusEnum.DELETE.getCode().equals(taskDO.getTaskStatus())) {
|
if (TaskStatusEnum.DELETE.getCode().equals(taskDO.getTaskStatus())) {
|
||||||
|
|
||||||
eventBus.post(new DeleteTaskEvent(taskDO));
|
eventBus.post(new DeleteTaskEvent(taskDO));
|
||||||
log.info("从数据库中查询到一个删除任务,发出一个同步事件:" + taskDO);
|
log.info("从数据库中查询到一个删除任务,发出一个同步事件:{}", taskDO);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
|
@ -17,7 +17,6 @@ import com.alibaba.nacossync.extension.event.SpecialSyncEvent;
|
||||||
import com.alibaba.nacossync.extension.event.SpecialSyncEventBus;
|
import com.alibaba.nacossync.extension.event.SpecialSyncEventBus;
|
||||||
import com.google.common.eventbus.EventBus;
|
import com.google.common.eventbus.EventBus;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.boot.CommandLineRunner;
|
import org.springframework.boot.CommandLineRunner;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
@ -33,20 +32,25 @@ import java.util.concurrent.TimeUnit;
|
||||||
@Service
|
@Service
|
||||||
public class SpecialSyncEventTimer implements CommandLineRunner {
|
public class SpecialSyncEventTimer implements CommandLineRunner {
|
||||||
|
|
||||||
@Autowired
|
private final SpecialSyncEventBus specialSyncEventBus;
|
||||||
private SpecialSyncEventBus specialSyncEventBus;
|
|
||||||
|
|
||||||
@Autowired
|
private final EventBus eventBus;
|
||||||
private EventBus eventBus;
|
|
||||||
|
|
||||||
@Autowired
|
private final ScheduledExecutorService scheduledExecutorService;
|
||||||
private ScheduledExecutorService scheduledExecutorService;
|
|
||||||
|
public SpecialSyncEventTimer(SpecialSyncEventBus specialSyncEventBus, EventBus eventBus,
|
||||||
|
ScheduledExecutorService scheduledExecutorService) {
|
||||||
|
this.specialSyncEventBus = specialSyncEventBus;
|
||||||
|
this.eventBus = eventBus;
|
||||||
|
this.scheduledExecutorService = scheduledExecutorService;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void run(String... args) throws Exception {
|
public void run(String... args) throws Exception {
|
||||||
|
|
||||||
scheduledExecutorService.scheduleWithFixedDelay(new SpecialSyncEventTimer.SpecialSyncEventThread(), 0, 3000,
|
scheduledExecutorService.scheduleWithFixedDelay(new SpecialSyncEventTimer.SpecialSyncEventThread(), 0, 3000,
|
||||||
TimeUnit.MILLISECONDS);
|
TimeUnit.MILLISECONDS);
|
||||||
|
log.info("SpecialSyncEventTimer has started successfully");
|
||||||
}
|
}
|
||||||
|
|
||||||
private class SpecialSyncEventThread implements Runnable {
|
private class SpecialSyncEventThread implements Runnable {
|
||||||
|
@ -58,9 +62,9 @@ public class SpecialSyncEventTimer implements CommandLineRunner {
|
||||||
allSpecialSyncEvent.stream()
|
allSpecialSyncEvent.stream()
|
||||||
.filter(specialSyncEvent -> TaskStatusEnum.SYNC.getCode()
|
.filter(specialSyncEvent -> TaskStatusEnum.SYNC.getCode()
|
||||||
.equals(specialSyncEvent.getTaskDO().getTaskStatus()))
|
.equals(specialSyncEvent.getTaskDO().getTaskStatus()))
|
||||||
.forEach(specialSyncEvent -> eventBus.post(specialSyncEvent));
|
.forEach(eventBus::post);
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.warn("SpecialSyncEventThread Exception", e);
|
log.error("Exception occurred while processing special sync events", e);
|
||||||
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -0,0 +1,104 @@
|
||||||
|
package com.alibaba.nacossync.util;
|
||||||
|
|
||||||
|
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
|
||||||
|
import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
|
import com.google.common.base.Stopwatch;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.Collections;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.concurrent.*;
|
||||||
|
import java.util.function.Consumer;
|
||||||
|
|
||||||
|
@Slf4j
|
||||||
|
public class BatchTaskExecutor {
|
||||||
|
|
||||||
|
private static final int MAX_THREAD_NUM = 200;
|
||||||
|
private static final ExecutorService executorService = Executors.newFixedThreadPool(MAX_THREAD_NUM);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Batch operation method
|
||||||
|
*
|
||||||
|
* @param items Task list
|
||||||
|
* @param operation Operation to be executed
|
||||||
|
*/
|
||||||
|
public static void batchOperation(List<TaskDO> items, Consumer<TaskDO> operation) {
|
||||||
|
Stopwatch stopwatch = Stopwatch.createStarted();
|
||||||
|
|
||||||
|
List<Tuple<Integer, List<TaskDO>>> taskGroupList = averageAssign(items, MAX_THREAD_NUM);
|
||||||
|
|
||||||
|
// Create a CompletableFuture for each task group
|
||||||
|
CompletableFuture<?>[] futures = taskGroupList.stream().map(tuple -> CompletableFuture.runAsync(() -> {
|
||||||
|
for (TaskDO taskDO : tuple.getT2()) {
|
||||||
|
try {
|
||||||
|
// Add timeout control for each task to avoid long-running tasks
|
||||||
|
CompletableFuture.runAsync(() -> operation.accept(taskDO), executorService)
|
||||||
|
.orTimeout(5, TimeUnit.SECONDS) // Task timeout set to 5 seconds
|
||||||
|
.exceptionally(ex -> {
|
||||||
|
log.error("Task execution timed out: {}", taskDO.getServiceName(), ex);
|
||||||
|
return null;
|
||||||
|
}).join();
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Error occurred during task execution: {}", taskDO.getServiceName(), e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, executorService)).toArray(CompletableFuture[]::new);
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Wait for all tasks to complete
|
||||||
|
CompletableFuture.allOf(futures).join();
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.error("Error occurred during sync operation", e);
|
||||||
|
} finally {
|
||||||
|
log.info("Total sync tasks: {}, Execution time: {} ms", items.size(), stopwatch.elapsed(TimeUnit.MILLISECONDS));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Divide a list into n sublists, mainly implemented by offset
|
||||||
|
* @param source collection to be divided
|
||||||
|
* @param limit maximum value
|
||||||
|
* @return list after division
|
||||||
|
* @param <T> object type
|
||||||
|
*/
|
||||||
|
private static <T> List<Tuple<Integer, List<T>>> averageAssign(List<T> source, int limit) {
|
||||||
|
if (CollectionUtils.isEmpty(source)) {
|
||||||
|
return Collections.emptyList();
|
||||||
|
}
|
||||||
|
|
||||||
|
int size = source.size();
|
||||||
|
int listCount = (int) Math.ceil((double) size / limit); // Calculate the number of sublists
|
||||||
|
int remainder = size % listCount; // Calculate the number of remaining elements after even distribution
|
||||||
|
List<Tuple<Integer, List<T>>> result = new ArrayList<>(listCount); // Initialize the result list with the expected size
|
||||||
|
|
||||||
|
for (int i = 0, assigned = 0; i < listCount; i++) {
|
||||||
|
int sublistSize = size / listCount + (remainder-- > 0 ? 1 : 0); // Determine the size of each sublist, distribute remaining elements
|
||||||
|
List<T> sublist = new ArrayList<>(source.subList(assigned, assigned + sublistSize)); // Create the sublist
|
||||||
|
result.add(Tuple.of(i, sublist)); // Add the sublist to the result
|
||||||
|
assigned += sublistSize; // Update the assigned index
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Shutdown the executor service to avoid resource leakage
|
||||||
|
*/
|
||||||
|
public static void shutdown() {
|
||||||
|
executorService.shutdown();
|
||||||
|
try {
|
||||||
|
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||||
|
executorService.shutdownNow();
|
||||||
|
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
|
||||||
|
log.error("Executor service did not terminate");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (InterruptedException ie) {
|
||||||
|
executorService.shutdownNow();
|
||||||
|
Thread.currentThread().interrupt();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
|
@ -1,19 +0,0 @@
|
||||||
package com.alibaba.nacossync.util;
|
|
||||||
|
|
||||||
import java.util.ArrayList;
|
|
||||||
import java.util.Collection;
|
|
||||||
import java.util.Iterator;
|
|
||||||
|
|
||||||
public abstract class Collections {
|
|
||||||
|
|
||||||
public static Collection subtract(final Collection a, final Collection b) {
|
|
||||||
ArrayList list = new ArrayList( a );
|
|
||||||
for (Iterator it = b.iterator(); it.hasNext();) {
|
|
||||||
list.remove(it.next());
|
|
||||||
}
|
|
||||||
return list;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
}
|
|
|
@ -1,15 +1,3 @@
|
||||||
/*
|
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE
|
|
||||||
* file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
|
|
||||||
* to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the
|
|
||||||
* License. You may obtain a copy of the License at
|
|
||||||
*
|
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
*
|
|
||||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
||||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
||||||
* specific language governing permissions and limitations under the License.
|
|
||||||
*/
|
|
||||||
package com.alibaba.nacossync.util;
|
package com.alibaba.nacossync.util;
|
||||||
|
|
||||||
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
|
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
|
||||||
|
@ -20,16 +8,21 @@ import java.util.Map;
|
||||||
import java.util.stream.Collectors;
|
import java.util.stream.Collectors;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author paderlol
|
* Utility class for handling Consul metadata.
|
||||||
* @date: 2019-04-25 00:01
|
|
||||||
*/
|
*/
|
||||||
public class ConsulUtils {
|
public class ConsulUtils {
|
||||||
public static Map<String, String> transferMetadata(List<String> tags) {
|
public static Map<String, String> transferMetadata(List<String> tags) {
|
||||||
Map<String, String> metadata = new HashMap<>();
|
if (CollectionUtils.isEmpty(tags)) {
|
||||||
if (!CollectionUtils.isEmpty(tags)) {
|
return new HashMap<>();
|
||||||
return tags.stream().filter(tag -> tag.split("=", -1).length == 2).map(tag -> tag.split("=", -1))
|
|
||||||
.collect(Collectors.toMap(tagSplitArray -> tagSplitArray[0], tagSplitArray -> tagSplitArray[1]));
|
|
||||||
}
|
}
|
||||||
return metadata;
|
|
||||||
|
return tags.stream()
|
||||||
|
.map(tag -> tag.split("=", -1))
|
||||||
|
.filter(tagArray -> tagArray.length == 2)
|
||||||
|
.collect(Collectors.toMap(
|
||||||
|
tagArray -> tagArray[0],
|
||||||
|
tagArray -> tagArray[1],
|
||||||
|
(existing, replacement) -> existing // In case of duplicate keys, keep the existing value
|
||||||
|
));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -20,7 +20,6 @@ import org.apache.commons.lang3.StringUtils;
|
||||||
import java.math.BigDecimal;
|
import java.math.BigDecimal;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Map;
|
import java.util.Map;
|
||||||
import java.util.function.Predicate;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author paderlol
|
* @author paderlol
|
||||||
|
@ -51,43 +50,49 @@ public final class DubboConstants {
|
||||||
public static final String ALL_SERVICE_NAME_PATTERN = "*";
|
public static final String ALL_SERVICE_NAME_PATTERN = "*";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* if Dubbo version greater than 2.7.2, service name is providers:interface:version:
|
* Creates a service name based on Dubbo version compatibility.
|
||||||
* if Dubbo version less than 2.7.2, service name is providers:interface:version
|
* if Dubbo version greater than 2.7.2, service name is providers:interface:version:
|
||||||
* @param queryParam
|
* if Dubbo version less than 2.7.2, service name is providers:interface:version
|
||||||
* @return
|
*
|
||||||
|
* @param queryParam the query parameters that include keys such as interface, version, group, etc.
|
||||||
|
* @return the constructed service name string
|
||||||
*/
|
*/
|
||||||
public static String createServiceName(Map<String, String> queryParam) {
|
public static String createServiceName(Map<String, String> queryParam) {
|
||||||
|
|
||||||
String group = queryParam.get(GROUP_KEY);
|
String group = queryParam.get(GROUP_KEY);
|
||||||
String release = queryParam.get(RELEASE_KEY);
|
String release = queryParam.get(RELEASE_KEY);
|
||||||
Predicate<String> isBlankGroup = StringUtils::isBlank;
|
|
||||||
Predicate<String> isNotBlankRelease = StringUtils::isNotBlank;
|
|
||||||
String serviceName = Joiner.on(SEPARATOR_KEY).skipNulls().join(CATALOG_KEY, queryParam.get(INTERFACE_KEY),
|
|
||||||
queryParam.get(VERSION_KEY), group);
|
|
||||||
|
|
||||||
//TODO The code here is to deal with service metadata format problems caused by dubbo version incompatibility
|
String baseServiceName = Joiner.on(SEPARATOR_KEY).skipNulls().join(CATALOG_KEY, queryParam.get(INTERFACE_KEY),
|
||||||
if (isBlankGroup.test(group) && isNotBlankRelease.test(release)) {
|
queryParam.get(VERSION_KEY), group);
|
||||||
|
|
||||||
|
if (StringUtils.isBlank(group) && StringUtils.isNotBlank(release)) {
|
||||||
List<String> versions = Splitter.on(RELEASE_SEPARATOR_KEY).splitToList(release);
|
List<String> versions = Splitter.on(RELEASE_SEPARATOR_KEY).splitToList(release);
|
||||||
if (!CollectionUtils.isEmpty(versions) && versions.size() >= DUBBO_VERSION_INDEX) {
|
if (!CollectionUtils.isEmpty(versions) && versions.size() >= DUBBO_VERSION_INDEX) {
|
||||||
String firstVersion = versions.get(0);
|
String firstVersion = versions.get(0);
|
||||||
String secondVersion = versions.get(1);
|
String secondVersion = versions.get(1);
|
||||||
if (DUBBO_VERSION_INDEX == Integer.parseInt(firstVersion)) {
|
BigDecimal bigDecimal = new BigDecimal(Joiner.on(RELEASE_SEPARATOR_KEY).join(secondVersion,
|
||||||
if (MIDDLE_DUBBO_VERSION_INDEX <= versions.size()) {
|
versions.size() > 2 ? versions.get(2) : "0"));
|
||||||
String thirdVersion = versions.get(2);
|
if (isVersionRequiresSeparator(firstVersion, secondVersion, bigDecimal)) {
|
||||||
BigDecimal bigDecimal =
|
baseServiceName = baseServiceName.concat(SEPARATOR_KEY);
|
||||||
new BigDecimal(Joiner.on(RELEASE_SEPARATOR_KEY).join(secondVersion, thirdVersion));
|
|
||||||
if (bigDecimal.compareTo(COMPARE_NUMBER) > 0) {
|
|
||||||
serviceName = serviceName.concat(SEPARATOR_KEY);
|
|
||||||
}
|
|
||||||
} else if (versions.size() == DUBBO_VERSION_INDEX && Integer.parseInt(secondVersion) > 7) {
|
|
||||||
serviceName = serviceName.concat(SEPARATOR_KEY);
|
|
||||||
}
|
|
||||||
} else if (MIN_DUBBO_VERSION < Integer.parseInt(firstVersion)) {
|
|
||||||
serviceName = serviceName.concat(SEPARATOR_KEY);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return serviceName;
|
return baseServiceName;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checks if the version requires a separator to be appended to the service name.
|
||||||
|
*
|
||||||
|
* @param firstVersion the major version
|
||||||
|
* @param secondVersion the minor version
|
||||||
|
* @param bigDecimal the version number as BigDecimal
|
||||||
|
* @return true if separator should be added, otherwise false
|
||||||
|
*/
|
||||||
|
private static boolean isVersionRequiresSeparator(String firstVersion, String secondVersion, BigDecimal bigDecimal) {
|
||||||
|
int majorVersion = Integer.parseInt(firstVersion);
|
||||||
|
int minorVersion = Integer.parseInt(secondVersion);
|
||||||
|
|
||||||
|
return (DUBBO_VERSION_INDEX == majorVersion && (MIDDLE_DUBBO_VERSION_INDEX <= minorVersion ||
|
||||||
|
bigDecimal.compareTo(COMPARE_NUMBER) > 0)) || (MIN_DUBBO_VERSION < majorVersion);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -0,0 +1,11 @@
|
||||||
|
package com.alibaba.nacossync.util;
|
||||||
|
|
||||||
|
import com.alibaba.nacos.api.common.Constants;
|
||||||
|
import org.apache.commons.lang3.StringUtils;
|
||||||
|
|
||||||
|
public class NacosUtils {
|
||||||
|
|
||||||
|
public static String getGroupNameOrDefault(String groupName) {
|
||||||
|
return StringUtils.defaultIfBlank(groupName, Constants.DEFAULT_GROUP);
|
||||||
|
}
|
||||||
|
}
|
|
@ -1,19 +1,3 @@
|
||||||
/*
|
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
|
||||||
* contributor license agreements. See the NOTICE file distributed with
|
|
||||||
* this work for additional information regarding copyright ownership.
|
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
|
||||||
* (the "License"); you may not use this file except in compliance with
|
|
||||||
* the License. You may obtain a copy of the License at
|
|
||||||
*
|
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
*
|
|
||||||
* Unless required by applicable law or agreed to in writing, software
|
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
* See the License for the specific language governing permissions and
|
|
||||||
* limitations under the License.
|
|
||||||
*/
|
|
||||||
package com.alibaba.nacossync.util;
|
package com.alibaba.nacossync.util;
|
||||||
|
|
||||||
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
import com.alibaba.nacossync.constant.ClusterTypeEnum;
|
||||||
|
@ -22,135 +6,139 @@ import com.alibaba.nacossync.pojo.model.TaskDO;
|
||||||
import com.alibaba.nacossync.pojo.request.ClusterAddRequest;
|
import com.alibaba.nacossync.pojo.request.ClusterAddRequest;
|
||||||
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
import com.alibaba.nacossync.pojo.request.TaskAddRequest;
|
||||||
import com.google.common.base.Joiner;
|
import com.google.common.base.Joiner;
|
||||||
|
import org.apache.commons.lang3.StringUtils;
|
||||||
|
|
||||||
import java.io.UnsupportedEncodingException;
|
|
||||||
import java.net.Inet4Address;
|
import java.net.Inet4Address;
|
||||||
import java.net.InetAddress;
|
import java.net.InetAddress;
|
||||||
import java.net.NetworkInterface;
|
import java.net.NetworkInterface;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
import java.security.MessageDigest;
|
import java.security.MessageDigest;
|
||||||
import java.security.NoSuchAlgorithmException;
|
import java.security.NoSuchAlgorithmException;
|
||||||
import java.util.Enumeration;
|
import java.util.Enumeration;
|
||||||
import java.util.UUID;
|
import java.util.UUID;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author NacosSync
|
* Utility class for various operations in SkyWalker.
|
||||||
* @version $Id: SkyWalkerUtil.java, v 0.1 2018-09-26 AM12:10 NacosSync Exp $$
|
*/
|
||||||
*/
|
|
||||||
public class SkyWalkerUtil {
|
public class SkyWalkerUtil {
|
||||||
|
|
||||||
|
private static final String SEPARATOR = ":";
|
||||||
|
private static final String MD5_ALGORITHM = "MD5";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
* Generates an MD5 hash for the given string.
|
||||||
*
|
*
|
||||||
* Gets the string md5
|
* @param value The string to be encrypted.
|
||||||
* @param value
|
* @return The encrypted string, or an empty string if encryption fails.
|
||||||
* @return
|
|
||||||
*/
|
*/
|
||||||
public static String StringToMd5(String value) {
|
public static String stringToMd5(String value) {
|
||||||
{
|
if (StringUtils.isBlank(value)) {
|
||||||
try {
|
return "";
|
||||||
MessageDigest md5 = MessageDigest.getInstance("MD5");
|
}
|
||||||
md5.update(value.getBytes("UTF-8"));
|
try {
|
||||||
byte[] encryption = md5.digest();
|
MessageDigest md5 = MessageDigest.getInstance(MD5_ALGORITHM);
|
||||||
StringBuffer strBuf = new StringBuffer();
|
byte[] encryption = md5.digest(value.getBytes(StandardCharsets.UTF_8));
|
||||||
for (int i = 0; i < encryption.length; i++) {
|
StringBuilder strBuf = new StringBuilder();
|
||||||
if (Integer.toHexString(0xff & encryption[i]).length() == 1) {
|
for (byte b : encryption) {
|
||||||
strBuf.append("0").append(Integer.toHexString(0xff & encryption[i]));
|
strBuf.append(String.format("%02x", b));
|
||||||
} else {
|
|
||||||
strBuf.append(Integer.toHexString(0xff & encryption[i]));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return strBuf.toString();
|
|
||||||
} catch (NoSuchAlgorithmException e) {
|
|
||||||
return "";
|
|
||||||
} catch (UnsupportedEncodingException e) {
|
|
||||||
return "";
|
|
||||||
}
|
}
|
||||||
|
return strBuf.toString();
|
||||||
|
} catch (NoSuchAlgorithmException e) {
|
||||||
|
return "";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The rules of generating taskId
|
* Generates a task ID based on the given TaskAddRequest.
|
||||||
* @param addTaskRequest
|
*
|
||||||
* @return
|
* @param addTaskRequest The TaskAddRequest containing task details.
|
||||||
|
* @return The generated task ID.
|
||||||
*/
|
*/
|
||||||
public static String generateTaskId(TaskAddRequest addTaskRequest) {
|
public static String generateTaskId(TaskAddRequest addTaskRequest) {
|
||||||
|
|
||||||
return generateTaskId(addTaskRequest.getServiceName(), addTaskRequest.getGroupName(),
|
return generateTaskId(addTaskRequest.getServiceName(), addTaskRequest.getGroupName(),
|
||||||
addTaskRequest.getSourceClusterId(), addTaskRequest.getDestClusterId());
|
addTaskRequest.getSourceClusterId(), addTaskRequest.getDestClusterId());
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The rules of generating taskId
|
* Generates a task ID based on the given parameters.
|
||||||
*
|
*
|
||||||
* @return
|
* @param serviceName The service name.
|
||||||
|
* @param groupName The group name.
|
||||||
|
* @param sourceClusterId The source cluster ID.
|
||||||
|
* @param destClusterId The destination cluster ID.
|
||||||
|
* @return The generated task ID.
|
||||||
*/
|
*/
|
||||||
public static String generateTaskId(String serviceName, String groupName,
|
public static String generateTaskId(String serviceName, String groupName,
|
||||||
String sourceClusterId, String destClusterId) {
|
String sourceClusterId, String destClusterId) {
|
||||||
|
String rawId = String.join(SkyWalkerConstants.UNDERLINE, serviceName, groupName, sourceClusterId, destClusterId);
|
||||||
StringBuilder sb = new StringBuilder();
|
return stringToMd5(rawId);
|
||||||
|
|
||||||
sb.append(serviceName);
|
|
||||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
|
||||||
sb.append(groupName);
|
|
||||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
|
||||||
sb.append(sourceClusterId);
|
|
||||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
|
||||||
sb.append(destClusterId);
|
|
||||||
return SkyWalkerUtil.StringToMd5(sb.toString());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 生成集群clusterId的规则
|
* Generates a cluster ID based on the given ClusterAddRequest.
|
||||||
*
|
*
|
||||||
* @param addClusterRequest
|
* @param addClusterRequest The ClusterAddRequest containing cluster details.
|
||||||
* @return
|
* @return The generated cluster ID.
|
||||||
*/
|
*/
|
||||||
public static String generateClusterId(ClusterAddRequest addClusterRequest) {
|
public static String generateClusterId(ClusterAddRequest addClusterRequest) {
|
||||||
|
String rawId = String.join(SkyWalkerConstants.UNDERLINE, addClusterRequest.getClusterName(), addClusterRequest.getClusterType());
|
||||||
StringBuilder sb = new StringBuilder();
|
return stringToMd5(rawId);
|
||||||
sb.append(addClusterRequest.getClusterName());
|
|
||||||
sb.append(SkyWalkerConstants.UNDERLINE);
|
|
||||||
sb.append(addClusterRequest.getClusterType());
|
|
||||||
|
|
||||||
return SkyWalkerUtil.StringToMd5(sb.toString());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Avoid getting a return address
|
* Gets the local IP address, avoiding loopback addresses.
|
||||||
* @return
|
*
|
||||||
* @throws Exception
|
* @return The local IP address.
|
||||||
|
* @throws Exception If an error occurs while fetching the IP address.
|
||||||
*/
|
*/
|
||||||
public static String getLocalIp() throws Exception {
|
public static String getLocalIp() throws Exception {
|
||||||
|
|
||||||
InetAddress addr = InetAddress.getLocalHost();
|
InetAddress addr = InetAddress.getLocalHost();
|
||||||
String localIp = addr.getHostAddress();
|
if (!addr.isLoopbackAddress()) {
|
||||||
if (addr.isLoopbackAddress()) {
|
return addr.getHostAddress();
|
||||||
Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
|
}
|
||||||
while (interfaces.hasMoreElements()) {
|
Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
|
||||||
NetworkInterface in = interfaces.nextElement();
|
while (interfaces.hasMoreElements()) {
|
||||||
Enumeration<InetAddress> addrs = in.getInetAddresses();
|
NetworkInterface networkInterface = interfaces.nextElement();
|
||||||
while (addrs.hasMoreElements()) {
|
Enumeration<InetAddress> addrs = networkInterface.getInetAddresses();
|
||||||
InetAddress address = addrs.nextElement();
|
while (addrs.hasMoreElements()) {
|
||||||
if (!address.isLoopbackAddress() && address instanceof Inet4Address) {
|
InetAddress address = addrs.nextElement();
|
||||||
localIp = address.getHostAddress();
|
if (!address.isLoopbackAddress() && address instanceof Inet4Address) {
|
||||||
}
|
return address.getHostAddress();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return localIp;
|
return addr.getHostAddress();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generates a synchronization key based on source and destination cluster types.
|
||||||
|
*
|
||||||
|
* @param sourceClusterType The source cluster type.
|
||||||
|
* @param destClusterType The destination cluster type.
|
||||||
|
* @return The generated synchronization key.
|
||||||
|
*/
|
||||||
public static String generateSyncKey(ClusterTypeEnum sourceClusterType, ClusterTypeEnum destClusterType) {
|
public static String generateSyncKey(ClusterTypeEnum sourceClusterType, ClusterTypeEnum destClusterType) {
|
||||||
|
return Joiner.on(SEPARATOR).join(sourceClusterType.getCode(), destClusterType.getCode());
|
||||||
return Joiner.on(":").join(sourceClusterType.getCode(), destClusterType.getCode());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Gets the operation ID from the given TaskDO.
|
||||||
|
*
|
||||||
|
* @param taskDO The TaskDO containing the operation ID.
|
||||||
|
* @return The operation ID.
|
||||||
|
*/
|
||||||
public static String getOperationId(TaskDO taskDO) {
|
public static String getOperationId(TaskDO taskDO) {
|
||||||
|
|
||||||
return taskDO.getOperationId();
|
return taskDO.getOperationId();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generates a unique operation ID.
|
||||||
|
*
|
||||||
|
* @return The generated operation ID.
|
||||||
|
*/
|
||||||
public static String generateOperationId() {
|
public static String generateOperationId() {
|
||||||
|
|
||||||
return UUID.randomUUID().toString();
|
return UUID.randomUUID().toString();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -13,19 +13,23 @@
|
||||||
package com.alibaba.nacossync.util;
|
package com.alibaba.nacossync.util;
|
||||||
|
|
||||||
import com.google.common.base.Joiner;
|
import com.google.common.base.Joiner;
|
||||||
import com.google.common.collect.Maps;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
|
||||||
import java.io.File;
|
|
||||||
import java.io.UnsupportedEncodingException;
|
|
||||||
import java.net.URLDecoder;
|
import java.net.URLDecoder;
|
||||||
import java.net.URLEncoder;
|
import java.net.URLEncoder;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
import java.util.HashMap;
|
import java.util.HashMap;
|
||||||
import java.util.Map;
|
import java.util.Map;
|
||||||
import java.util.regex.Matcher;
|
import java.util.regex.Matcher;
|
||||||
import java.util.regex.Pattern;
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
import static com.alibaba.nacossync.util.DubboConstants.*;
|
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_PATH_FORMAT;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.DUBBO_URL_FORMAT;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_IP_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.INSTANCE_PORT_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.INTERFACE_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.PROTOCOL_KEY;
|
||||||
|
import static com.alibaba.nacossync.util.DubboConstants.ZOOKEEPER_SEPARATOR;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author paderlol
|
* @author paderlol
|
||||||
|
@ -35,24 +39,26 @@ import static com.alibaba.nacossync.util.DubboConstants.*;
|
||||||
public final class StringUtils {
|
public final class StringUtils {
|
||||||
|
|
||||||
private static final Pattern KVP_PATTERN = Pattern
|
private static final Pattern KVP_PATTERN = Pattern
|
||||||
.compile("([_.a-zA-Z0-9][-_.a-zA-Z0-9]*)[=](.*)");
|
.compile("([_.a-zA-Z0-9][-_.a-zA-Z0-9]*)[=](.*)");
|
||||||
private static final Pattern IP_PORT_PATTERN = Pattern
|
private static final Pattern IP_PORT_PATTERN = Pattern
|
||||||
.compile(".*/(.*)://(\\d+\\.\\d+\\.\\d+\\.\\d+):(\\d+)");
|
.compile(".*/(.*)://(\\d+\\.\\d+\\.\\d+\\.\\d+):(\\d+)");
|
||||||
private static final Pattern DUBBO_PROVIDER_PATTERN = Pattern
|
private static final Pattern DUBBO_PROVIDER_PATTERN = Pattern
|
||||||
.compile("/dubbo/(.*)/providers/(.*)");
|
.compile("/dubbo/(.*)/providers/(.*)");
|
||||||
|
public static final int INDEX_NOT_FOUND = -1;
|
||||||
|
public static final String EMPTY = "";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* parse key-value pair.
|
* parse key-value pair.
|
||||||
*
|
*
|
||||||
* @param str string.
|
* @param str string.
|
||||||
* @param itemSeparator item separator.
|
* @param itemSeparator item separator.
|
||||||
* @return key-value map;
|
* @return key-value map;
|
||||||
*/
|
*/
|
||||||
private static Map<String, String> parseKeyValuePair(String str, String itemSeparator) {
|
private static Map<String, String> parseKeyValuePair(String str, String itemSeparator) {
|
||||||
String[] tmp = str.split(itemSeparator);
|
String[] tmp = str.split(itemSeparator);
|
||||||
Map<String, String> map = new HashMap<String, String>(tmp.length);
|
Map<String, String> map = new HashMap<>(tmp.length);
|
||||||
for (int i = 0; i < tmp.length; i++) {
|
for (String s : tmp) {
|
||||||
Matcher matcher = KVP_PATTERN.matcher(tmp[i]);
|
Matcher matcher = KVP_PATTERN.matcher(s);
|
||||||
if (!matcher.matches()) {
|
if (!matcher.matches()) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
@ -68,17 +74,14 @@ public final class StringUtils {
|
||||||
* @return Parameters instance.
|
* @return Parameters instance.
|
||||||
*/
|
*/
|
||||||
public static Map<String, String> parseQueryString(String qs) {
|
public static Map<String, String> parseQueryString(String qs) {
|
||||||
try {
|
|
||||||
String decodePath = URLDecoder.decode(qs, "UTF-8");
|
|
||||||
if (isEmpty(qs)) {
|
|
||||||
return new HashMap<>();
|
|
||||||
}
|
|
||||||
return parseKeyValuePair(decodePath, "\\&");
|
|
||||||
|
|
||||||
} catch (UnsupportedEncodingException e) {
|
String decodePath = URLDecoder.decode(qs, StandardCharsets.UTF_8);
|
||||||
log.warn("parse query string failed", e);
|
if (isEmpty(decodePath)) {
|
||||||
return Maps.newHashMap();
|
return new HashMap<>();
|
||||||
}
|
}
|
||||||
|
decodePath = substringAfter(decodePath, "?");
|
||||||
|
return parseKeyValuePair(decodePath, "&");
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -93,48 +96,78 @@ public final class StringUtils {
|
||||||
|
|
||||||
public static Map<String, String> parseIpAndPortString(String path) {
|
public static Map<String, String> parseIpAndPortString(String path) {
|
||||||
|
|
||||||
try {
|
String decodePath = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
||||||
String decodePath = URLDecoder.decode(path, "UTF-8");
|
Matcher matcher = IP_PORT_PATTERN.matcher(decodePath);
|
||||||
Matcher matcher = IP_PORT_PATTERN.matcher(decodePath);
|
// extract the ones that match the rules
|
||||||
// extract the ones that match the rules
|
Map<String, String> instanceMap = new HashMap<>(3);
|
||||||
Map<String, String> instanceMap = new HashMap<>(3);
|
while (matcher.find()) {
|
||||||
while (matcher.find()) {
|
// protocol
|
||||||
// protocol
|
instanceMap.put(PROTOCOL_KEY, matcher.group(1));
|
||||||
instanceMap.put(PROTOCOL_KEY, matcher.group(1));
|
// ip address
|
||||||
// ip address
|
instanceMap.put(INSTANCE_IP_KEY, matcher.group(2));
|
||||||
instanceMap.put(INSTANCE_IP_KEY, matcher.group(2));
|
// port
|
||||||
// port
|
instanceMap.put(INSTANCE_PORT_KEY, matcher.group(3));
|
||||||
instanceMap.put(INSTANCE_PORT_KEY, matcher.group(3));
|
break;
|
||||||
break;
|
|
||||||
}
|
|
||||||
return instanceMap;
|
|
||||||
} catch (UnsupportedEncodingException e) {
|
|
||||||
log.warn("parse query string failed", e);
|
|
||||||
return Maps.newHashMap();
|
|
||||||
}
|
}
|
||||||
|
return instanceMap;
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* <p>Gets the substring after the first occurrence of a separator.
|
||||||
|
* The separator is not returned.</p>
|
||||||
|
*
|
||||||
|
* <p>A {@code null} string input will return {@code null}.
|
||||||
|
* An empty ("") string input will return the empty string. A {@code null} separator will return the empty string if
|
||||||
|
* the input string is not {@code null}.</p>
|
||||||
|
*
|
||||||
|
* <p>If nothing is found, the empty string is returned.</p>
|
||||||
|
*
|
||||||
|
* <pre>
|
||||||
|
* StringUtils.substringAfter(null, *) = null
|
||||||
|
* StringUtils.substringAfter("", *) = ""
|
||||||
|
* StringUtils.substringAfter(*, null) = ""
|
||||||
|
* StringUtils.substringAfter("abc", "a") = "bc"
|
||||||
|
* StringUtils.substringAfter("abcba", "b") = "cba"
|
||||||
|
* StringUtils.substringAfter("abc", "c") = ""
|
||||||
|
* StringUtils.substringAfter("abc", "d") = ""
|
||||||
|
* StringUtils.substringAfter("abc", "") = "abc"
|
||||||
|
* </pre>
|
||||||
|
*
|
||||||
|
* @param str the String to get a substring from, may be null
|
||||||
|
* @param separator the String to search for, may be null
|
||||||
|
* @return the substring after the first occurrence of the separator, {@code null} if null String input
|
||||||
|
* @since 2.0
|
||||||
|
*/
|
||||||
|
public static String substringAfter(final String str, final String separator) {
|
||||||
|
if (isEmpty(str)) {
|
||||||
|
return str;
|
||||||
|
}
|
||||||
|
if (separator == null) {
|
||||||
|
return EMPTY;
|
||||||
|
}
|
||||||
|
final int pos = str.indexOf(separator);
|
||||||
|
if (pos == INDEX_NOT_FOUND) {
|
||||||
|
return EMPTY;
|
||||||
|
}
|
||||||
|
return str.substring(pos + 1);
|
||||||
|
}
|
||||||
|
|
||||||
public static String convertDubboProvidersPath(String interfaceName) {
|
public static String convertDubboProvidersPath(String interfaceName) {
|
||||||
return String.format(DUBBO_PATH_FORMAT, interfaceName);
|
return String.format(DUBBO_PATH_FORMAT, interfaceName);
|
||||||
}
|
}
|
||||||
|
|
||||||
public static String convertDubboFullPathForZk(Map<String, String> metaData, String providersPath, String ip,
|
public static String convertDubboFullPathForZk(Map<String, String> metaData, String providersPath, String ip,
|
||||||
int port) {
|
int port) {
|
||||||
try {
|
String urlParam = Joiner.on("&").withKeyValueSeparator("=").join(metaData);
|
||||||
String urlParam = Joiner.on("&").withKeyValueSeparator("=").join(metaData);
|
String instanceUrl = String.format(DUBBO_URL_FORMAT, metaData.get(PROTOCOL_KEY), ip, port,
|
||||||
String instanceUrl = String.format(DUBBO_URL_FORMAT, metaData.get(PROTOCOL_KEY), ip, port,
|
metaData.get(INTERFACE_KEY), urlParam);
|
||||||
metaData.get(INTERFACE_KEY), urlParam);
|
|
||||||
|
|
||||||
return Joiner.on(ZOOKEEPER_SEPARATOR).join(providersPath, URLEncoder.encode(instanceUrl, "UTF-8"));
|
return Joiner.on(ZOOKEEPER_SEPARATOR).join(providersPath, URLEncoder.encode(instanceUrl, StandardCharsets.UTF_8));
|
||||||
} catch (UnsupportedEncodingException e) {
|
|
||||||
log.warn("convert Dubbo full path", e);
|
|
||||||
return "";
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public static boolean isDubboProviderPath(String path) {
|
public static boolean isDubboProviderPath(String path) {
|
||||||
return DUBBO_PROVIDER_PATTERN.matcher(path).matches();
|
return DUBBO_PROVIDER_PATTERN.matcher(path).matches();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -0,0 +1,207 @@
|
||||||
|
/*
|
||||||
|
* Copyright (c) 2016-2021 VMware Inc. or its affiliates, All Rights Reserved.
|
||||||
|
*
|
||||||
|
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
* you may not use this file except in compliance with the License.
|
||||||
|
* You may obtain a copy of the License at
|
||||||
|
*
|
||||||
|
* https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
*
|
||||||
|
* Unless required by applicable law or agreed to in writing, software
|
||||||
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
* See the License for the specific language governing permissions and
|
||||||
|
* limitations under the License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package com.alibaba.nacossync.util;
|
||||||
|
|
||||||
|
import reactor.util.annotation.NonNull;
|
||||||
|
import reactor.util.annotation.Nullable;
|
||||||
|
import reactor.util.function.Tuple2;
|
||||||
|
import reactor.util.function.Tuples;
|
||||||
|
|
||||||
|
import java.io.Serializable;
|
||||||
|
import java.util.Arrays;
|
||||||
|
import java.util.Collections;
|
||||||
|
import java.util.Iterator;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Objects;
|
||||||
|
import java.util.function.Function;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A tuple that holds two non-null values.
|
||||||
|
*
|
||||||
|
* @param <T1> The type of the first non-null value held by this tuple
|
||||||
|
* @param <T2> The type of the second non-null value held by this tuple
|
||||||
|
* @author Jon Brisbin
|
||||||
|
* @author Stephane Maldini
|
||||||
|
*/
|
||||||
|
@SuppressWarnings("rawtypes")
|
||||||
|
public class Tuple<T1, T2> implements Iterable<Object>, Serializable {
|
||||||
|
|
||||||
|
private static final long serialVersionUID = -3518082018884860684L;
|
||||||
|
|
||||||
|
@NonNull
|
||||||
|
final T1 t1;
|
||||||
|
|
||||||
|
@NonNull
|
||||||
|
final T2 t2;
|
||||||
|
|
||||||
|
Tuple(T1 t1, T2 t2) {
|
||||||
|
this.t1 = Objects.requireNonNull(t1, "t1");
|
||||||
|
this.t2 = Objects.requireNonNull(t2, "t2");
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Type-safe way to get the first object of this {@link Tuples}.
|
||||||
|
*
|
||||||
|
* @return The first object
|
||||||
|
*/
|
||||||
|
public T1 getT1() {
|
||||||
|
return t1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Type-safe way to get the second object of this {@link Tuples}.
|
||||||
|
*
|
||||||
|
* @return The second object
|
||||||
|
*/
|
||||||
|
public T2 getT2() {
|
||||||
|
return t2;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Map the left-hand part (T1) of this {@link Tuple} into a different value and type, keeping the right-hand part
|
||||||
|
* (T2).
|
||||||
|
*
|
||||||
|
* @param mapper the mapping {@link Function} for the left-hand part
|
||||||
|
* @param <R> the new type for the left-hand part
|
||||||
|
* @return a new {@link Tuple2} with a different left (T1) value
|
||||||
|
*/
|
||||||
|
public <R> Tuple<R, T2> mapT1(Function<T1, R> mapper) {
|
||||||
|
return new Tuple<>(mapper.apply(t1), t2);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Map the right-hand part (T2) of this {@link Tuple} into a different value and type, keeping the left-hand part
|
||||||
|
* (T1).
|
||||||
|
*
|
||||||
|
* @param mapper the mapping {@link Function} for the right-hand part
|
||||||
|
* @param <R> the new type for the right-hand part
|
||||||
|
* @return a new {@link Tuple2} with a different right (T2) value
|
||||||
|
*/
|
||||||
|
public <R> Tuple<T1, R> mapT2(Function<T2, R> mapper) {
|
||||||
|
return new Tuple<>(t1, mapper.apply(t2));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get the object at the given index.
|
||||||
|
*
|
||||||
|
* @param index The index of the object to retrieve. Starts at 0.
|
||||||
|
* @return The object or {@literal null} if out of bounds.
|
||||||
|
*/
|
||||||
|
@Nullable
|
||||||
|
public Object get(int index) {
|
||||||
|
switch (index) {
|
||||||
|
case 0:
|
||||||
|
return t1;
|
||||||
|
case 1:
|
||||||
|
return t2;
|
||||||
|
default:
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Turn this {@code Tuple} into a {@link List List<Object>}. The list isn't tied to this Tuple but is a
|
||||||
|
* <strong>copy</strong> with limited mutability ({@code add} and {@code remove} are not supported, but {@code set}
|
||||||
|
* is).
|
||||||
|
*
|
||||||
|
* @return A copy of the tuple as a new {@link List List<Object>}.
|
||||||
|
*/
|
||||||
|
public List<Object> toList() {
|
||||||
|
return Arrays.asList(toArray());
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Turn this {@code Tuple} into a plain {@code Object[]}. The array isn't tied to this Tuple but is a
|
||||||
|
* <strong>copy</strong>.
|
||||||
|
*
|
||||||
|
* @return A copy of the tuple as a new {@link Object Object[]}.
|
||||||
|
*/
|
||||||
|
public Object[] toArray() {
|
||||||
|
return new Object[] {t1, t2};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Return an <strong>immutable</strong> {@link Iterator Iterator<Object>} around the content of this
|
||||||
|
* {@code Tuple}.
|
||||||
|
*
|
||||||
|
* @return An unmodifiable {@link Iterator} over the elements in this Tuple.
|
||||||
|
* @implNote As an {@link Iterator} is always tied to its {@link Iterable} source by definition, the iterator cannot
|
||||||
|
* be mutable without the iterable also being mutable. Since {@link Tuples} are <strong>immutable</strong>, so is
|
||||||
|
* the {@link Iterator} returned by this method.
|
||||||
|
*/
|
||||||
|
@Override
|
||||||
|
public Iterator<Object> iterator() {
|
||||||
|
return Collections.unmodifiableList(toList()).iterator();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public boolean equals(@Nullable Object o) {
|
||||||
|
if (this == o) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (o == null || getClass() != o.getClass()) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
Tuple<?, ?> tuple = (Tuple<?, ?>) o;
|
||||||
|
|
||||||
|
return t1.equals(tuple.t1) && t2.equals(tuple.t2);
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public int hashCode() {
|
||||||
|
int result = size();
|
||||||
|
result = 31 * result + t1.hashCode();
|
||||||
|
result = 31 * result + t2.hashCode();
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Return the number of elements in this {@literal Tuples}.
|
||||||
|
*
|
||||||
|
* @return The size of this {@literal Tuples}.
|
||||||
|
*/
|
||||||
|
public int size() {
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A Tuple String representation is the comma separated list of values, enclosed in square brackets.
|
||||||
|
*
|
||||||
|
* @return the Tuple String representation
|
||||||
|
*/
|
||||||
|
@Override
|
||||||
|
public final String toString() {
|
||||||
|
Object[] values = toArray();
|
||||||
|
StringBuilder sb = new StringBuilder();
|
||||||
|
for (int i = 0; i < values.length; i++) {
|
||||||
|
Object t = values[i];
|
||||||
|
if (i != 0) {
|
||||||
|
sb.append(',');
|
||||||
|
}
|
||||||
|
if (t != null) {
|
||||||
|
sb.append(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return sb.insert(0, '[').append(']').toString();
|
||||||
|
}
|
||||||
|
|
||||||
|
public static <T1, T2> Tuple<T1, T2> of(T1 t1, T2 t2) {
|
||||||
|
return new Tuple<>(t1, t2);
|
||||||
|
}
|
||||||
|
}
|
|
@ -5,11 +5,11 @@ spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
|
||||||
spring.jpa.hibernate.ddl-auto=update
|
spring.jpa.hibernate.ddl-auto=update
|
||||||
spring.jpa.properties.hibernate.show_sql=false
|
spring.jpa.properties.hibernate.show_sql=false
|
||||||
|
|
||||||
|
spring.cloud.discovery.enabled=false
|
||||||
|
|
||||||
|
|
||||||
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/nacos_sync?characterEncoding=utf8
|
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/nacos_sync?characterEncoding=utf8
|
||||||
spring.datasource.username=root
|
spring.datasource.username=root
|
||||||
spring.datasource.password=root
|
spring.datasource.password=root
|
||||||
|
|
||||||
management.endpoints.web.exposure.include=*
|
management.endpoints.web.exposure.include=*
|
||||||
management.endpoint.health.show-details=always
|
management.endpoint.health.show-details=always
|
||||||
|
|
|
@ -4,12 +4,13 @@
|
||||||
<springProperty name="logPath" scope="context" source="nacosSync.logs.path" defaultValue="${nacosSync.home}/logs"/>
|
<springProperty name="logPath" scope="context" source="nacosSync.logs.path" defaultValue="${nacosSync.home}/logs"/>
|
||||||
<property name="LOG_HOME" value="${logPath}" />
|
<property name="LOG_HOME" value="${logPath}" />
|
||||||
<appender name="FILEINFO" class="ch.qos.logback.core.rolling.RollingFileAppender">
|
<appender name="FILEINFO" class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||||||
<file>${LOG_HOME}/nacosSync.log</file>
|
<file>${LOG_HOME}/nacos-sync.log</file>
|
||||||
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
|
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
|
||||||
<fileNamePattern>${LOG_HOME}/log-nacosSync-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
|
<fileNamePattern>${LOG_HOME}/log-nacos-sync-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
|
||||||
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
|
<maxFileSize>200MB</maxFileSize>
|
||||||
<maxFileSize>200MB</maxFileSize>
|
<maxHistory>7</maxHistory>
|
||||||
</timeBasedFileNamingAndTriggeringPolicy>
|
<totalSizeCap>2GB</totalSizeCap>
|
||||||
|
<cleanHistoryOnStart>true</cleanHistoryOnStart>
|
||||||
</rollingPolicy>
|
</rollingPolicy>
|
||||||
<append>true</append>
|
<append>true</append>
|
||||||
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
|
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
|
||||||
|
@ -25,7 +26,6 @@
|
||||||
</appender>
|
</appender>
|
||||||
|
|
||||||
<logger name="com.alibaba.nacossync" level="INFO"/>
|
<logger name="com.alibaba.nacossync" level="INFO"/>
|
||||||
|
|
||||||
<logger name="org.hibernate.type.descriptor.sql.BasicBinder" level="INFO"/>
|
<logger name="org.hibernate.type.descriptor.sql.BasicBinder" level="INFO"/>
|
||||||
<logger name="org.hibernate.type.descriptor.sql.BasicExtractor" level="INFO"/>
|
<logger name="org.hibernate.type.descriptor.sql.BasicExtractor" level="INFO"/>
|
||||||
<logger name="org.hibernate.SQL" level="INFO"/>
|
<logger name="org.hibernate.SQL" level="INFO"/>
|
||||||
|
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
@ -7,8 +7,8 @@
|
||||||
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
<meta http-equiv="X-UA-Compatible" content="ie=edge">
|
||||||
<title>Nacos-Sync</title>
|
<title>Nacos-Sync</title>
|
||||||
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
<link rel="shortcut icon" href="//www.aliyun.com/favicon.ico" type="image/x-icon">
|
||||||
<link href="./css/main.68bce23a.css" rel="stylesheet"></head>
|
<link href="./css/main.e2917886.css" rel="stylesheet"></head>
|
||||||
<body>
|
<body>
|
||||||
<div id="root"></div>
|
<div id="root"></div>
|
||||||
<script type="text/javascript" src="./js/main.1cd0c600.js"></script></body>
|
<script type="text/javascript" src="./js/main.b73436b5.js"></script></body>
|
||||||
</html>
|
</html>
|
||||||
|
|
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue