Remove mirrors configuration

Signed-off-by: Fan Shang <2444576154@qq.com>

.github/copilot-instructions.md

@@ -0,0 +1,250 @@
+# GitHub Copilot Instructions for Nydus
+
+## Project Overview
+
+Nydus is a high-performance container image service that implements a content-addressable file system on the RAFS format. It enhances the OCI image specification by enabling on-demand loading, chunk-level deduplication, and improved container startup performance.
+
+### Key Components
+
+- **nydusd**: User-space daemon that processes FUSE/fscache/virtiofs messages and serves Nydus images
+- **nydus-image**: CLI tool to convert OCI image layers to Nydus format
+- **nydusify**: Tool to convert entire OCI images to Nydus format with registry integration
+- **nydusctl**: CLI client for managing and querying nydusd daemon
+- **nydus-service**: Library crate for integrating Nydus services into other projects
+
+## Architecture Guidelines
+
+### Crate Structure
+```
+- api/          # Nydus Image Service APIs and data structures
+- builder/      # Image building and conversion logic
+- rafs/         # RAFS filesystem implementation
+- service/      # Daemon and service management framework
+- storage/      # Core storage subsystem with backends and caching
+- utils/        # Common utilities and helper functions
+- src/bin/      # Binary executables (nydusd, nydus-image, nydusctl)
+```
+
+### Key Technologies
+- **Language**: Rust with memory safety focus
+- **Filesystems**: FUSE, virtiofs, EROFS, fscache
+- **Storage Backends**: Registry, OSS, S3, LocalFS, HTTP proxy
+- **Compression**: LZ4, Gzip, Zstd
+- **Async Runtime**: Tokio (current thread for io-uring compatibility)
+
+## Code Style and Patterns
+
+### Rust Conventions
+- Use `#![deny(warnings)]` in all binary crates
+- Follow standard Rust naming conventions (snake_case, PascalCase)
+- Prefer `anyhow::Result` for error handling in applications
+- Use custom error types with `thiserror` for libraries
+- Apply `#[macro_use]` for frequently used external crates like `log`
+- Always format the code with `cargo fmt`
+- Use `clippy` for linting and follow its suggestions
+
+### Error Handling
+```rust
+// Prefer anyhow for applications
+use anyhow::{bail, Context, Result};
+
+// Use custom error types for libraries
+use thiserror::Error;
+
+#[derive(Error, Debug)]
+pub enum NydusError {
+    #[error("Invalid arguments: {0}")]
+    InvalidArguments(String),
+    #[error("IO error: {0}")]
+    Io(#[from] std::io::Error),
+}
+```
+
+### Logging Patterns
+- Use structured logging with appropriate levels (trace, debug, info, warn, error)
+- Include context in error messages: `.with_context(|| "description")`
+- Use `info!`, `warn!`, `error!` macros consistently
+
+### Configuration Management
+- Use `serde` for JSON configuration serialization/deserialization
+- Support both file-based and environment variable configuration
+- Validate configurations at startup with clear error messages
+- Follow the `ConfigV2` pattern for versioned configurations
+
+## Development Guidelines
+
+### Storage Backend Development
+- When implementing new storage backends:
+- - Implement the `BlobBackend` trait
+- - Support timeout, retry, and connection management
+- - Add configuration in the backend config structure
+- - Consider proxy support for high availability
+- - Implement proper error handling and logging  
+
+### Daemon Service Development
+- Use the `NydusDaemon` trait for service implementations
+- Support save/restore for hot upgrade functionality
+- Implement proper state machine transitions
+- Use `DaemonController` for lifecycle management
+
+### RAFS Filesystem Features
+- Support both RAFS v5 and v6 formats
+- Implement chunk-level deduplication
+- Handle prefetch optimization for container startup
+- Support overlay filesystem operations
+- Maintain POSIX compatibility
+
+### API Development
+- Use versioned APIs (v1, v2) with backward compatibility
+- Implement HTTP endpoints with proper error handling
+- Support both Unix socket and TCP communication
+- Follow OpenAPI specification patterns
+
+## Testing Patterns
+
+### Unit Tests
+- Test individual functions and modules in isolation
+- Use `#[cfg(test)]` modules within source files
+- Mock external dependencies when necessary
+- Focus on error conditions and edge cases
+
+### Integration Tests
+- Place integration tests in `tests/` directory
+- Test complete workflows and component interactions
+- Use temporary directories for filesystem operations
+- Clean up resources properly in test teardown
+
+### Smoke Tests
+- Located in `smoke/` directory using Go
+- Test real-world scenarios with actual images
+- Verify performance and functionality
+- Use Bats framework for shell-based testing
+
+## Performance Considerations
+
+### I/O Optimization
+- Use async I/O patterns with Tokio
+- Implement prefetching for predictable access patterns
+- Optimize chunk size (default 1MB) for workload characteristics
+- Consider io-uring for high-performance scenarios
+
+### Memory Management
+- Use `Arc<T>` for shared ownership of large objects
+- Implement lazy loading for metadata structures
+- Consider memory mapping for large files
+- Profile memory usage in performance-critical paths
+
+### Caching Strategy
+- Implement blob caching with configurable backends
+- Support compression in cache to save space
+- Use chunk-level caching with efficient eviction policies
+- Consider cache warming strategies for frequently accessed data
+
+## Security Guidelines
+
+### Data Integrity
+- Implement end-to-end digest validation
+- Support multiple hash algorithms (SHA256, Blake3)
+- Verify chunk integrity on read operations
+- Detect and prevent supply chain attacks
+
+### Authentication
+- Support registry authentication (basic auth, bearer tokens)
+- Handle credential rotation and refresh
+- Implement secure credential storage
+- Support mutual TLS for backend connections
+
+## Specific Code Patterns
+
+### Configuration Loading
+```rust
+// Standard pattern for configuration loading
+let config = match config_path {
+    Some(path) => ConfigV2::from_file(path)?,
+    None => ConfigV2::default(),
+};
+
+// Environment variable override
+if let Ok(auth) = std::env::var("IMAGE_PULL_AUTH") {
+    config.update_registry_auth_info(&auth);
+}
+```
+
+### Daemon Lifecycle
+```rust
+// Standard daemon initialization pattern
+let daemon = create_daemon(config, build_info)?;
+DAEMON_CONTROLLER.set_daemon(daemon);
+
+// Event loop management
+if DAEMON_CONTROLLER.is_active() {
+    DAEMON_CONTROLLER.run_loop();
+}
+
+// Graceful shutdown
+DAEMON_CONTROLLER.shutdown();
+```
+
+### Blob Access Pattern
+```rust
+// Standard blob read pattern
+let mut bio = BlobIoDesc::new(blob_id, blob_address, blob_size, user_io);
+let blob_device = factory.get_device(&blob_info)?;
+blob_device.read(&mut bio)?;
+```
+
+## Documentation Standards
+
+### Code Documentation
+- Document all public APIs with `///` comments
+- Include examples in documentation
+- Document safety requirements for unsafe code
+- Explain complex algorithms and data structures
+
+### Architecture Documentation
+- Maintain design documents in `docs/` directory
+- Update documentation when adding new features
+- Include diagrams for complex interactions
+- Document configuration options comprehensively
+
+### Release Notes
+- Document breaking changes clearly
+- Include migration guides for major versions
+- Highlight performance improvements
+- List new features and bug fixes
+
+## Container and Cloud Native Patterns
+
+### OCI Compatibility
+- Maintain compatibility with OCI image spec
+- Support standard container runtimes (runc, Kata)
+- Implement proper layer handling and manifest generation
+- Support multi-architecture images
+
+### Kubernetes Integration
+- Design for Kubernetes CRI integration
+- Support containerd snapshotter pattern
+- Handle pod lifecycle events appropriately
+- Implement proper resource cleanup
+
+### Cloud Storage Integration
+- Support major cloud providers (AWS S3, Alibaba OSS)
+- Implement proper credential management
+- Handle network interruptions gracefully
+- Support cross-region replication patterns
+
+## Build and Release
+
+### Build Configuration
+- Use `Cargo.toml` workspace configuration
+- Support cross-compilation for multiple architectures
+- Implement proper feature flags for optional components
+- Use consistent dependency versioning
+
+### Release Process
+- Tag releases with semantic versioning
+- Generate release binaries for supported platforms
+- Update documentation with release notes
+- Validate release artifacts before publishing
+
+Remember to follow these guidelines when contributing to or working with the Nydus codebase. The project emphasizes performance, security, and compatibility with the broader container ecosystem.

.gitignore

@@ -10,3 +10,5 @@ go.work.sum
 dist/
 nydus-static/
 .goreleaser.yml
+metadata.db
+tests/texture/zran/233c72f2b6b698c07021c4da367cfe2dff4f049efbaa885ca0ff760ea297865a

Cargo.lock

@@ -926,7 +926,7 @@ dependencies = [
  "httpdate",
  "itoa",
  "pin-project-lite",
- "socket2 0.4.10",
+ "socket2 0.5.8",
  "tokio",
  "tower-service",
  "tracing",
@@ -1563,6 +1563,7 @@ dependencies = [
  "nydus-storage",
  "nydus-upgrade",
  "nydus-utils",
+ "procfs",
  "rust-fsm",
  "serde",
  "serde_json",
@@ -1824,6 +1825,31 @@ dependencies = [
  "unicode-ident",
 ]
 
+[[package]]
+name = "procfs"
+version = "0.17.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cc5b72d8145275d844d4b5f6d4e1eef00c8cd889edb6035c21675d1bb1f45c9f"
+dependencies = [
+ "bitflags 2.8.0",
+ "chrono",
+ "flate2",
+ "hex",
+ "procfs-core",
+ "rustix",
+]
+
+[[package]]
+name = "procfs-core"
+version = "0.17.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "239df02d8349b06fc07398a3a1697b06418223b1c7725085e801e7c0fc6a12ec"
+dependencies = [
+ "bitflags 2.8.0",
+ "chrono",
+ "hex",
+]
+
 [[package]]
 name = "quote"
 version = "1.0.38"

api/src/config.rs

@@ -519,9 +519,6 @@ pub struct OssConfig {
     /// Enable HTTP proxy for the read request.
     #[serde(default)]
     pub proxy: ProxyConfig,
-    /// Enable mirrors for the read request.
-    #[serde(default)]
-    pub mirrors: Vec<MirrorConfig>,
 }
 
 /// S3 configuration information to access blobs.
@@ -563,9 +560,6 @@ pub struct S3Config {
     /// Enable HTTP proxy for the read request.
     #[serde(default)]
     pub proxy: ProxyConfig,
-    /// Enable mirrors for the read request.
-    #[serde(default)]
-    pub mirrors: Vec<MirrorConfig>,
 }
 
 /// Http proxy configuration information to access blobs.
@@ -592,9 +586,6 @@ pub struct HttpProxyConfig {
     /// Enable HTTP proxy for the read request.
     #[serde(default)]
     pub proxy: ProxyConfig,
-    /// Enable mirrors for the read request.
-    #[serde(default)]
-    pub mirrors: Vec<MirrorConfig>,
 }
 
 /// Container registry configuration information to access blobs.
@@ -635,9 +626,6 @@ pub struct RegistryConfig {
     /// Enable HTTP proxy for the read request.
     #[serde(default)]
     pub proxy: ProxyConfig,
-    /// Enable mirrors for the read request.
-    #[serde(default)]
-    pub mirrors: Vec<MirrorConfig>,
 }
 
 /// Configuration information for blob cache manager.
@@ -930,41 +918,6 @@ impl Default for ProxyConfig {
     }
 }
 
-/// Configuration for registry mirror.
-#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
-pub struct MirrorConfig {
-    /// Mirror server URL, for example http://127.0.0.1:65001.
-    pub host: String,
-    /// Ping URL to check mirror server health.
-    #[serde(default)]
-    pub ping_url: String,
-    /// HTTP request headers to be passed to mirror server.
-    #[serde(default)]
-    pub headers: HashMap<String, String>,
-    /// Interval for mirror health checking, in seconds.
-    #[serde(default = "default_check_interval")]
-    pub health_check_interval: u64,
-    /// Maximum number of failures before marking a mirror as unusable.
-    #[serde(default = "default_failure_limit")]
-    pub failure_limit: u8,
-    /// Elapsed time to pause mirror health check when the request is inactive, in seconds.
-    #[serde(default = "default_check_pause_elapsed")]
-    pub health_check_pause_elapsed: u64,
-}
-
-impl Default for MirrorConfig {
-    fn default() -> Self {
-        Self {
-            host: String::new(),
-            headers: HashMap::new(),
-            health_check_interval: 5,
-            failure_limit: 5,
-            ping_url: String::new(),
-            health_check_pause_elapsed: 300,
-        }
-    }
-}
-
 /// Configuration information for a cached blob`.
 #[derive(Clone, Debug, Default, Deserialize, Eq, PartialEq, Serialize)]
 pub struct BlobCacheEntryConfigV2 {
@@ -1212,10 +1165,6 @@ fn default_check_pause_elapsed() -> u64 {
     300
 }
 
-fn default_failure_limit() -> u8 {
-    5
-}
-
 fn default_work_dir() -> String {
     ".".to_string()
 }
@@ -1886,11 +1835,6 @@ mod tests {
         fallback = true
         check_interval = 10
         use_http = true
-        [[backend.oss.mirrors]]
-        host = "http://127.0.0.1:65001"
-        ping_url = "http://127.0.0.1:65001/ping"
-        health_check_interval = 10
-        failure_limit = 10
         "#;
         let config: ConfigV2 = toml::from_str(content).unwrap();
         assert_eq!(config.version, 2);
@@ -1917,14 +1861,6 @@ mod tests {
         assert_eq!(oss.proxy.check_interval, 10);
         assert!(oss.proxy.fallback);
         assert!(oss.proxy.use_http);
-
-        assert_eq!(oss.mirrors.len(), 1);
-        let mirror = &oss.mirrors[0];
-        assert_eq!(mirror.host, "http://127.0.0.1:65001");
-        assert_eq!(mirror.ping_url, "http://127.0.0.1:65001/ping");
-        assert!(mirror.headers.is_empty());
-        assert_eq!(mirror.health_check_interval, 10);
-        assert_eq!(mirror.failure_limit, 10);
     }
 
     #[test]
@@ -1950,11 +1886,6 @@ mod tests {
         fallback = true
         check_interval = 10
         use_http = true
-        [[backend.registry.mirrors]]
-        host = "http://127.0.0.1:65001"
-        ping_url = "http://127.0.0.1:65001/ping"
-        health_check_interval = 10
-        failure_limit = 10
         "#;
         let config: ConfigV2 = toml::from_str(content).unwrap();
         assert_eq!(config.version, 2);
@@ -1983,14 +1914,6 @@ mod tests {
         assert_eq!(registry.proxy.check_interval, 10);
         assert!(registry.proxy.fallback);
         assert!(registry.proxy.use_http);
-
-        assert_eq!(registry.mirrors.len(), 1);
-        let mirror = &registry.mirrors[0];
-        assert_eq!(mirror.host, "http://127.0.0.1:65001");
-        assert_eq!(mirror.ping_url, "http://127.0.0.1:65001/ping");
-        assert!(mirror.headers.is_empty());
-        assert_eq!(mirror.health_check_interval, 10);
-        assert_eq!(mirror.failure_limit, 10);
     }
 
     #[test]
@@ -2397,15 +2320,6 @@ mod tests {
         assert!(res);
     }
 
-    #[test]
-    fn test_default_mirror_config() {
-        let cfg = MirrorConfig::default();
-        assert_eq!(cfg.host, "");
-        assert_eq!(cfg.health_check_interval, 5);
-        assert_eq!(cfg.failure_limit, 5);
-        assert_eq!(cfg.ping_url, "");
-    }
-
     #[test]
     fn test_config_v2_from_file() {
         let content = r#"version=2
@@ -2615,7 +2529,6 @@ mod tests {
     #[test]
     fn test_default_value() {
         assert!(default_true());
-        assert_eq!(default_failure_limit(), 5);
         assert_eq!(default_prefetch_batch_size(), 1024 * 1024);
         assert_eq!(default_prefetch_threads_count(), 8);
     }

api/src/error.rs

@@ -86,6 +86,8 @@ define_error_macro!(eother, std::io::Error::new(std::io::ErrorKind::Other, ""));
 
 #[cfg(test)]
 mod tests {
+    use std::io::{Error, ErrorKind};
+
     fn check_size(size: usize) -> std::io::Result<()> {
         if size > 0x1000 {
             return Err(einval!());
@@ -101,4 +103,150 @@ mod tests {
             std::io::Error::from_raw_os_error(libc::EINVAL).kind()
         );
     }
+
+    #[test]
+    fn test_make_error() {
+        let original_error = Error::new(ErrorKind::Other, "test error");
+        let debug_info = "debug information";
+        let file = "test.rs";
+        let line = 42;
+
+        let result_error = super::make_error(original_error, debug_info, file, line);
+        assert_eq!(result_error.kind(), ErrorKind::Other);
+    }
+
+    #[test]
+    fn test_libc_error_macros() {
+        // Test einval macro
+        let err = einval!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EINVAL).kind());
+
+        // Test enoent macro
+        let err = enoent!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::ENOENT).kind());
+
+        // Test ebadf macro
+        let err = ebadf!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EBADF).kind());
+
+        // Test eacces macro
+        let err = eacces!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EACCES).kind());
+
+        // Test enotdir macro
+        let err = enotdir!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::ENOTDIR).kind());
+
+        // Test eisdir macro
+        let err = eisdir!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EISDIR).kind());
+
+        // Test ealready macro
+        let err = ealready!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EALREADY).kind());
+
+        // Test enosys macro
+        let err = enosys!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::ENOSYS).kind());
+
+        // Test epipe macro
+        let err = epipe!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EPIPE).kind());
+
+        // Test eio macro
+        let err = eio!();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EIO).kind());
+    }
+
+    #[test]
+    fn test_libc_error_macros_with_context() {
+        let test_msg = "test context";
+
+        // Test einval macro with context
+        let err = einval!(test_msg);
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EINVAL).kind());
+
+        // Test enoent macro with context
+        let err = enoent!(test_msg);
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::ENOENT).kind());
+
+        // Test eio macro with context
+        let err = eio!(test_msg);
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EIO).kind());
+    }
+
+    #[test]
+    fn test_custom_error_macros() {
+        // Test last_error macro
+        let err = last_error!();
+        // We can't predict the exact error, but we can check it's a valid error
+        assert!(!err.to_string().is_empty());
+
+        // Test eother macro
+        let err = eother!();
+        assert_eq!(err.kind(), ErrorKind::Other);
+
+        // Test eother macro with context
+        let err = eother!("custom context");
+        assert_eq!(err.kind(), ErrorKind::Other);
+    }
+
+    fn test_bail_einval_function() -> std::io::Result<()> {
+        bail_einval!("test error message");
+    }
+
+    fn test_bail_eio_function() -> std::io::Result<()> {
+        bail_eio!("test error message");
+    }
+
+    #[test]
+    fn test_bail_macros() {
+        // Test bail_einval macro
+        let result = test_bail_einval_function();
+        assert!(result.is_err());
+        let err = result.unwrap_err();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EINVAL).kind());
+        // The error message format is controlled by the macro, so just check it's not empty
+        assert!(!err.to_string().is_empty());
+
+        // Test bail_eio macro
+        let result = test_bail_eio_function();
+        assert!(result.is_err());
+        let err = result.unwrap_err();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EIO).kind());
+        // The error message format is controlled by the macro, so just check it's not empty
+        assert!(!err.to_string().is_empty());
+    }
+
+    #[test]
+    fn test_bail_macros_with_formatting() {
+        fn test_bail_with_format(code: i32) -> std::io::Result<()> {
+            if code == 1 {
+                bail_einval!("error code: {}", code);
+            } else if code == 2 {
+                bail_eio!("I/O error with code: {}", code);
+            }
+            Ok(())
+        }
+
+        // Test bail_einval with formatting
+        let result = test_bail_with_format(1);
+        assert!(result.is_err());
+        let err = result.unwrap_err();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EINVAL).kind());
+        // The error message format is controlled by the macro, so just check it's not empty
+        assert!(!err.to_string().is_empty());
+
+        // Test bail_eio with formatting
+        let result = test_bail_with_format(2);
+        assert!(result.is_err());
+        let err = result.unwrap_err();
+        assert_eq!(err.kind(), Error::from_raw_os_error(libc::EIO).kind());
+        // The error message format is controlled by the macro, so just check it's not empty
+        assert!(!err.to_string().is_empty());
+
+        // Test success case
+        let result = test_bail_with_format(3);
+        assert!(result.is_ok());
+    }
 }

docs/nydusd.md

@@ -319,64 +319,8 @@ or
   }
 }
 ```
-
-The `HttpProxy` backend also supports the `Proxy` and `Mirrors` configurations for remote usage like the `Registry backend` described above.
-
-##### Enable Mirrors for Storage Backend (Recommend)
-
-Nydus is deeply integrated with [Dragonfly](https://d7y.io/) P2P mirror mode, please refer the [doc](https://d7y.io/docs/next/operations/integrations/container-runtime/nydus/) to learn how configuring Nydus to use Dragonfly.
-
-Add `device.backend.config.mirrors` field to enable mirrors for storage backend. The mirror can be a P2P distribution server or registry. If the request to mirror server failed, it will fall back to the original registry.
-Currently, the mirror mode is only tested in the registry backend, and in theory, the OSS backend also supports it.
-
-<font color='red'>!!</font> The `mirrors` field conflicts with `proxy` field.
-
-```
-{
-  "device": {
-    "backend": {
-      "type": "registry",
-      "config": {
-        "mirrors": [
-          {
-            // Mirror server URL (including scheme), e.g. Dragonfly dfdaemon server URL
-            "host": "http://dragonfly1.io:65001",
-            // Headers for mirror server
-            "headers": {
-              // For Dragonfly dfdaemon server URL, we need to specify "X-Dragonfly-Registry" (including scheme).
-              // When Dragonfly does not cache data, nydusd will pull it from "X-Dragonfly-Registry".
-              // If not set "X-Dragonfly-Registry", Dragonfly will pull data from proxy.registryMirror.url.
-              "X-Dragonfly-Registry": "https://index.docker.io"
-            },
-            // This URL endpoint is used to check the health of mirror server, and if the mirror is unhealthy, 
-            // the request will fallback to the next mirror or the original registry server. 
-            // Use $host/v2 as default if left empty.
-            "ping_url": "http://127.0.0.1:40901/server/ping",
-            // Interval time (s) to check and recover unavailable mirror. Use 5 as default if left empty.
-            "health_check_interval": 5,
-            // Failure counts before disabling this mirror. Use 5 as default if left empty.
-            "failure_limit": 5,
-            // Elapsed time to pause mirror health check when the request is inactive, in seconds.
-            // Use 300 as default if left empty.
-            "health_check_pause_elapsed": 300,
-          },
-          {
-            "host": "http://dragonfly2.io:65001",
-            "headers": {
-              "X-Dragonfly-Registry": "https://index.docker.io"
-            },
-          }
-        ],
-        ...
-      }
-    },
-    ...
-  },
-  ...
-}
-```
-
-
+-
+- The `HttpProxy` backend also supports the `Proxy` configuration for remote usage like the `Registry backend` described above.
 ##### Enable P2P Proxy for Storage Backend
 
 Add `device.backend.config.proxy` field to enable HTTP proxy for storage backend. For example, use P2P distribution service to reduce network workload and latency in large scale container cluster using [Dragonfly](https://d7y.io/) (enable centralized dfdaemon mode).

misc/configs/nydusd-blob-cache-entry-configuration-v2.toml

@@ -50,18 +50,6 @@ check_interval = 5
 # Replace URL to http to request source registry with proxy, and allow fallback to https if the proxy is unhealthy.
 use_http = false
 
-[[backend.oss.mirrors]]
-# Mirror server URL, for example http://127.0.0.1:65001.
-host = "http://127.0.0.1:65001"
-# Ping URL to check mirror server health.
-ping_url = "http://127.0.0.1:65001/ping"
-# HTTP request headers to be passed to mirror server.
-# headers =
-# Interval for mirror health checking, in seconds.
-health_check_interval = 5
-# Maximum number of failures before marking a mirror as unusable.
-failure_limit = 5
-
 [backend.registy]
 # Registry http scheme, either 'http' or 'https'
 scheme = "https"
@@ -99,18 +87,6 @@ check_interval = 5
 # Replace URL to http to request source registry with proxy, and allow fallback to https if the proxy is unhealthy.
 use_http = false
 
-[[backend.registry.mirrors]]
-# Mirror server URL, for example http://127.0.0.1:65001.
-host = "http://127.0.0.1:65001"
-# Ping URL to check mirror server health.
-ping_url = "http://127.0.0.1:65001/ping"
-# HTTP request headers to be passed to mirror server.
-# headers =
-# Interval for mirror health checking, in seconds.
-health_check_interval = 5
-# Maximum number of failures before marking a mirror as unusable.
-failure_limit = 5
-
 [cache]
 # Type of blob cache: "blobcache", "filecache", "fscache", "dummycache" or ""
 type = "filecache"

misc/configs/nydusd-blob-cache-entry.toml

@@ -55,18 +55,6 @@ check_interval = 5
 # Replace URL to http to request source registry with proxy, and allow fallback to https if the proxy is unhealthy.
 use_http = false
 
-[[config_v2.backend.oss.mirrors]]
-# Mirror server URL, for example http://127.0.0.1:65001.
-host = "http://127.0.0.1:65001"
-# Ping URL to check mirror server health.
-ping_url = "http://127.0.0.1:65001/ping"
-# HTTP request headers to be passed to mirror server.
-# headers =
-# Interval for mirror health checking, in seconds.
-health_check_interval = 5
-# Maximum number of failures before marking a mirror as unusable.
-failure_limit = 5
-
 [config_v2.backend.registry]
 # Registry http scheme, either 'http' or 'https'
 scheme = "https"
@@ -104,18 +92,6 @@ check_interval = 5
 # Replace URL to http to request source registry with proxy, and allow fallback to https if the proxy is unhealthy.
 use_http = false
 
-[[config_v2.backend.registry.mirrors]]
-# Mirror server URL, for example http://127.0.0.1:65001.
-host = "http://127.0.0.1:65001"
-# Ping URL to check mirror server health.
-ping_url = "http://127.0.0.1:65001/ping"
-# HTTP request headers to be passed to mirror server.
-# headers =
-# Interval for mirror health checking, in seconds.
-health_check_interval = 5
-# Maximum number of failures before marking a mirror as unusable.
-failure_limit = 5
-
 [config_v2.cache]
 # Type of blob cache: "blobcache", "filecache", "fscache", "dummycache" or ""
 type = "filecache"

misc/configs/nydusd-config-v2.toml

@@ -48,18 +48,6 @@ check_interval = 5
 # Replace URL to http to request source registry with proxy, and allow fallback to https if the proxy is unhealthy.
 use_http = false
 
-[[backend.oss.mirrors]]
-# Mirror server URL, for example http://127.0.0.1:65001.
-host = "http://127.0.0.1:65001"
-# Ping URL to check mirror server health.
-ping_url = "http://127.0.0.1:65001/ping"
-# HTTP request headers to be passed to mirror server.
-# headers =
-# Interval for mirror health checking, in seconds.
-health_check_interval = 5
-# Maximum number of failures before marking a mirror as unusable.
-failure_limit = 5
-
 [backend.registy]
 # Registry http scheme, either 'http' or 'https'
 scheme = "https"
@@ -97,17 +85,6 @@ check_interval = 5
 # Replace URL to http to request source registry with proxy, and allow fallback to https if the proxy is unhealthy.
 use_http = false
 
-[[backend.registry.mirrors]]
-# Mirror server URL, for example http://127.0.0.1:65001.
-host = "http://127.0.0.1:65001"
-# Ping URL to check mirror server health.
-ping_url = "http://127.0.0.1:65001/ping"
-# HTTP request headers to be passed to mirror server.
-# headers =
-# Interval for mirror health checking, in seconds.
-health_check_interval = 5
-# Maximum number of failures before marking a mirror as unusable.
-failure_limit = 5
 
 [cache]
 # Type of blob cache: "blobcache", "filecache", "fscache", "dummycache" or ""

misc/performance/snapshotter_config.toml

@@ -63,12 +63,6 @@ address = ":9110"
 [remote]
 convert_vpc_registry = false
 
-[remote.mirrors_config]
-# Snapshotter will overwrite daemon's mirrors configuration
-# if the values loaded from this driectory are not null before starting a daemon.
-# Set to "" or an empty directory to disable it.
-#dir = "/etc/nydus/certs.d"
-
 [remote.auth]
 # Fetch the private registry auth by listening to K8s API server
 enable_kubeconfig_keychain = false

rafs/src/metadata/cached_v5.rs

@@ -826,6 +826,7 @@ mod cached_tests {
     use crate::metadata::layout::{RafsXAttrs, RAFS_V5_ROOT_INODE};
     use crate::metadata::{
         RafsInode, RafsInodeWalkAction, RafsStore, RafsSuperBlock, RafsSuperInodes, RafsSuperMeta,
+        RAFS_MAX_NAME,
     };
     use crate::{BufWriter, RafsInodeExt, RafsIoRead, RafsIoReader};
     use vmm_sys_util::tempfile::TempFile;
@@ -1194,4 +1195,833 @@ mod cached_tests {
         assert!(info.is_compressed());
         assert!(!info.is_encrypted());
     }
+
+    #[test]
+    fn test_cached_inode_v5_validation_errors() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+
+        // Test invalid inode number (0)
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 0;
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test invalid nlink (0)
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 1;
+        inode.i_nlink = 0;
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test invalid parent for non-root inode
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 2;
+        inode.i_nlink = 1;
+        inode.i_parent = 0;
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test invalid name length
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 1;
+        inode.i_nlink = 1;
+        inode.i_name = OsString::from("a".repeat(RAFS_MAX_NAME + 1));
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test empty name
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 1;
+        inode.i_nlink = 1;
+        inode.i_name = OsString::new();
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test invalid parent inode (parent >= child for non-hardlink)
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 5;
+        inode.i_nlink = 1;
+        inode.i_parent = 10;
+        inode.i_name = OsString::from("test");
+        assert!(inode.validate(100, 1024).is_err());
+    }
+
+    #[test]
+    fn test_cached_inode_v5_file_type_validation() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+
+        // Test regular file with invalid chunk count
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 1;
+        inode.i_nlink = 1;
+        inode.i_name = OsString::from("test");
+        inode.i_mode = libc::S_IFREG as u32;
+        inode.i_size = 2048; // 2 chunks of 1024 bytes
+        inode.i_data = vec![]; // But no chunks
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test regular file with invalid block count
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 1;
+        inode.i_nlink = 1;
+        inode.i_name = OsString::from("test");
+        inode.i_mode = libc::S_IFREG as u32;
+        inode.i_size = 1024;
+        inode.i_blocks = 100; // Invalid block count
+        inode.i_data = vec![Arc::new(CachedChunkInfoV5::new())];
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test directory with invalid child index
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 5;
+        inode.i_nlink = 1;
+        inode.i_name = OsString::from("test_dir");
+        inode.i_mode = libc::S_IFDIR as u32;
+        inode.i_child_cnt = 1;
+        inode.i_child_idx = 3; // child_idx <= inode number is invalid
+        assert!(inode.validate(100, 1024).is_err());
+
+        // Test symlink with empty target
+        let mut inode = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        inode.i_ino = 1;
+        inode.i_nlink = 1;
+        inode.i_name = OsString::from("test_link");
+        inode.i_mode = libc::S_IFLNK as u32;
+        inode.i_target = OsString::new(); // Empty target
+        assert!(inode.validate(100, 1024).is_err());
+    }
+
+    #[test]
+    fn test_cached_inode_v5_file_type_checks() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut inode = CachedInodeV5::new(blob_table, meta);
+
+        // Test block device
+        inode.i_mode = libc::S_IFBLK as u32;
+        assert!(inode.is_blkdev());
+        assert!(!inode.is_chrdev());
+        assert!(!inode.is_sock());
+        assert!(!inode.is_fifo());
+        assert!(!inode.is_dir());
+        assert!(!inode.is_symlink());
+        assert!(!inode.is_reg());
+
+        // Test character device
+        inode.i_mode = libc::S_IFCHR as u32;
+        assert!(!inode.is_blkdev());
+        assert!(inode.is_chrdev());
+        assert!(!inode.is_sock());
+        assert!(!inode.is_fifo());
+
+        // Test socket
+        inode.i_mode = libc::S_IFSOCK as u32;
+        assert!(!inode.is_blkdev());
+        assert!(!inode.is_chrdev());
+        assert!(inode.is_sock());
+        assert!(!inode.is_fifo());
+
+        // Test FIFO
+        inode.i_mode = libc::S_IFIFO as u32;
+        assert!(!inode.is_blkdev());
+        assert!(!inode.is_chrdev());
+        assert!(!inode.is_sock());
+        assert!(inode.is_fifo());
+
+        // Test hardlink detection
+        inode.i_mode = libc::S_IFREG as u32;
+        inode.i_nlink = 2;
+        assert!(inode.is_hardlink());
+
+        inode.i_mode = libc::S_IFDIR as u32;
+        inode.i_nlink = 2;
+        assert!(!inode.is_hardlink()); // Directories are not considered hardlinks
+    }
+
+    #[test]
+    fn test_cached_inode_v5_xattr_operations() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut inode = CachedInodeV5::new(blob_table, meta);
+
+        // Test xattr flag
+        inode.i_flags = RafsInodeFlags::XATTR;
+        assert!(inode.has_xattr());
+
+        // Add some xattrs
+        inode
+            .i_xattr
+            .insert(OsString::from("user.test1"), vec![1, 2, 3]);
+        inode
+            .i_xattr
+            .insert(OsString::from("user.test2"), vec![4, 5, 6]);
+
+        // Test get_xattr
+        let value = inode.get_xattr(OsStr::new("user.test1")).unwrap();
+        assert_eq!(value, Some(vec![1, 2, 3]));
+
+        let value = inode.get_xattr(OsStr::new("user.nonexistent")).unwrap();
+        assert_eq!(value, None);
+
+        // Test get_xattrs
+        let xattrs = inode.get_xattrs().unwrap();
+        assert_eq!(xattrs.len(), 2);
+        assert!(xattrs.contains(&b"user.test1".to_vec()));
+        assert!(xattrs.contains(&b"user.test2".to_vec()));
+    }
+
+    #[test]
+    fn test_cached_inode_v5_symlink_operations() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut inode = CachedInodeV5::new(blob_table, meta);
+
+        // Test non-symlink
+        inode.i_mode = libc::S_IFREG as u32;
+        assert!(inode.get_symlink().is_err());
+        assert_eq!(inode.get_symlink_size(), 0);
+
+        // Test symlink
+        inode.i_mode = libc::S_IFLNK as u32;
+        inode.i_target = OsString::from("/path/to/target");
+
+        let target = inode.get_symlink().unwrap();
+        assert_eq!(target, OsString::from("/path/to/target"));
+        assert_eq!(inode.get_symlink_size(), "/path/to/target".len() as u16);
+    }
+
+    #[test]
+    fn test_cached_inode_v5_child_operations() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut parent = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        parent.i_ino = 1;
+        parent.i_mode = libc::S_IFDIR as u32;
+        parent.i_child_cnt = 2;
+
+        // Create child inodes
+        let mut child1 = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        child1.i_ino = 2;
+        child1.i_name = OsString::from("child_b");
+        child1.i_mode = libc::S_IFREG as u32;
+
+        let mut child2 = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        child2.i_ino = 3;
+        child2.i_name = OsString::from("child_a");
+        child2.i_mode = libc::S_IFREG as u32;
+
+        // Add children (they should be sorted by name)
+        parent.add_child(Arc::new(child1));
+        parent.add_child(Arc::new(child2));
+
+        // Test children are sorted
+        assert_eq!(parent.i_child[0].i_name, OsString::from("child_a"));
+        assert_eq!(parent.i_child[1].i_name, OsString::from("child_b"));
+
+        // Test get_child_by_name
+        let child = parent.get_child_by_name(OsStr::new("child_a")).unwrap();
+        assert_eq!(child.ino(), 3);
+
+        assert!(parent.get_child_by_name(OsStr::new("nonexistent")).is_err());
+
+        // Test get_child_by_index
+        let child = parent.get_child_by_index(0).unwrap();
+        assert_eq!(child.ino(), 3);
+
+        let child = parent.get_child_by_index(1).unwrap();
+        assert_eq!(child.ino(), 2);
+
+        assert!(parent.get_child_by_index(2).is_err());
+
+        // Test get_child_count
+        assert_eq!(parent.get_child_count(), 2);
+    }
+
+    #[test]
+    fn test_cached_inode_v5_walk_children() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut parent = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        parent.i_ino = 1;
+        parent.i_mode = libc::S_IFDIR as u32;
+        parent.i_child_cnt = 1;
+
+        let mut child = CachedInodeV5::new(blob_table, meta);
+        child.i_ino = 2;
+        child.i_name = OsString::from("test_child");
+        parent.add_child(Arc::new(child));
+
+        // Test walking from offset 0 (should see ".", "..", and "test_child")
+        let mut entries = Vec::new();
+        parent
+            .walk_children_inodes(0, &mut |_node, name, ino, offset| {
+                entries.push((name, ino, offset));
+                Ok(RafsInodeWalkAction::Continue)
+            })
+            .unwrap();
+
+        assert_eq!(entries.len(), 3);
+        assert_eq!(entries[0].0, OsString::from("."));
+        assert_eq!(entries[0].1, 1); // parent inode
+        assert_eq!(entries[1].0, OsString::from(".."));
+        assert_eq!(entries[1].1, 1); // root case
+        assert_eq!(entries[2].0, OsString::from("test_child"));
+        assert_eq!(entries[2].1, 2);
+
+        // Test walking from offset 1 (should skip ".")
+        let mut entries = Vec::new();
+        parent
+            .walk_children_inodes(1, &mut |_node, name, ino, _offset| {
+                entries.push((name, ino));
+                Ok(RafsInodeWalkAction::Continue)
+            })
+            .unwrap();
+
+        assert_eq!(entries.len(), 2);
+        assert_eq!(entries[0].0, OsString::from(".."));
+        assert_eq!(entries[1].0, OsString::from("test_child"));
+
+        // Test early break
+        let mut count = 0;
+        parent
+            .walk_children_inodes(0, &mut |_node, _name, _ino, _offset| {
+                count += 1;
+                if count == 1 {
+                    Ok(RafsInodeWalkAction::Break)
+                } else {
+                    Ok(RafsInodeWalkAction::Continue)
+                }
+            })
+            .unwrap();
+
+        assert_eq!(count, 1);
+    }
+
+    #[test]
+    fn test_cached_inode_v5_chunk_operations() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut inode = CachedInodeV5::new(blob_table, meta);
+
+        // Add some chunks
+        let mut chunk1 = CachedChunkInfoV5::new();
+        chunk1.index = 0;
+        chunk1.file_offset = 0;
+        chunk1.uncompressed_size = 1024;
+
+        let mut chunk2 = CachedChunkInfoV5::new();
+        chunk2.index = 1;
+        chunk2.file_offset = 1024;
+        chunk2.uncompressed_size = 1024;
+
+        inode.i_data.push(Arc::new(chunk1));
+        inode.i_data.push(Arc::new(chunk2));
+
+        // Note: get_chunk_count() currently returns i_child_cnt, not i_data.len()
+        // This appears to be a bug in the implementation, but we test current behavior
+        assert_eq!(inode.get_chunk_count(), 0); // i_child_cnt is 0 by default
+
+        // Test get_chunk_info
+        let chunk = inode.get_chunk_info(0).unwrap();
+        assert_eq!(chunk.uncompressed_size(), 1024);
+
+        let chunk = inode.get_chunk_info(1).unwrap();
+        assert_eq!(chunk.uncompressed_size(), 1024);
+
+        assert!(inode.get_chunk_info(2).is_err());
+
+        // Test actual data length
+        assert_eq!(inode.i_data.len(), 2);
+    }
+
+    #[test]
+    fn test_cached_inode_v5_collect_descendants() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+
+        // Create a directory structure
+        let mut root = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        root.i_ino = 1;
+        root.i_mode = libc::S_IFDIR as u32;
+        root.i_size = 0;
+
+        let mut subdir = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        subdir.i_ino = 2;
+        subdir.i_mode = libc::S_IFDIR as u32;
+        subdir.i_size = 0;
+
+        let mut file1 = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        file1.i_ino = 3;
+        file1.i_mode = libc::S_IFREG as u32;
+        file1.i_size = 1024;
+
+        let mut file2 = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        file2.i_ino = 4;
+        file2.i_mode = libc::S_IFREG as u32;
+        file2.i_size = 0; // Empty file should be skipped
+
+        let mut file3 = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        file3.i_ino = 5;
+        file3.i_mode = libc::S_IFREG as u32;
+        file3.i_size = 2048;
+
+        // Build structure: root -> [subdir, file1, file2], subdir -> [file3]
+        subdir.i_child.push(Arc::new(file3));
+        root.i_child.push(Arc::new(subdir));
+        root.i_child.push(Arc::new(file1));
+        root.i_child.push(Arc::new(file2));
+
+        let mut descendants = Vec::new();
+        root.collect_descendants_inodes(&mut descendants).unwrap();
+
+        // Should collect file1 (non-empty) and file3 (from subdirectory)
+        // file2 should be skipped because it's empty
+        assert_eq!(descendants.len(), 2);
+        let inodes: Vec<u64> = descendants.iter().map(|d| d.ino()).collect();
+        assert!(inodes.contains(&3)); // file1
+        assert!(inodes.contains(&5)); // file3
+        assert!(!inodes.contains(&4)); // file2 (empty)
+
+        // Test with non-directory
+        let file = CachedInodeV5::new(blob_table, meta);
+        let mut descendants = Vec::new();
+        assert!(file.collect_descendants_inodes(&mut descendants).is_err());
+    }
+
+    #[test]
+    fn test_cached_chunk_info_v5_detailed() {
+        let mut info = CachedChunkInfoV5::new();
+        info.block_id = Arc::new(RafsDigest::from_buf("test".as_bytes(), Algorithm::Blake3));
+        info.blob_index = 42;
+        info.index = 100;
+        info.file_offset = 2048;
+        info.compressed_offset = 1024;
+        info.uncompressed_offset = 3072;
+        info.compressed_size = 512;
+        info.uncompressed_size = 1024;
+        info.flags = BlobChunkFlags::COMPRESSED | BlobChunkFlags::HAS_CRC32;
+        info.crc32 = 0x12345678;
+
+        // Test basic properties
+        assert_eq!(info.id(), 100);
+        assert!(!info.is_batch());
+        assert!(info.is_compressed());
+        assert!(!info.is_encrypted());
+        assert!(info.has_crc32());
+        assert_eq!(info.crc32(), 0x12345678);
+
+        // Test getters
+        assert_eq!(info.blob_index(), 42);
+        assert_eq!(info.compressed_offset(), 1024);
+        assert_eq!(info.compressed_size(), 512);
+        assert_eq!(info.uncompressed_offset(), 3072);
+        assert_eq!(info.uncompressed_size(), 1024);
+
+        // Test V5-specific getters
+        assert_eq!(info.index(), 100);
+        assert_eq!(info.file_offset(), 2048);
+        assert_eq!(
+            info.flags(),
+            BlobChunkFlags::COMPRESSED | BlobChunkFlags::HAS_CRC32
+        );
+
+        // Test CRC32 without flag
+        info.flags = BlobChunkFlags::COMPRESSED;
+        assert!(!info.has_crc32());
+        assert_eq!(info.crc32(), 0);
+
+        // Test as_base
+        let base_info = info.as_base();
+        assert_eq!(base_info.blob_index(), 42);
+        assert!(base_info.is_compressed());
+    }
+
+    #[test]
+    fn test_cached_superblock_v5_inode_management() {
+        let md = RafsSuperMeta::default();
+        let mut sb = CachedSuperBlockV5::new(md, false);
+
+        // Test empty superblock
+        assert_eq!(sb.get_max_ino(), RAFS_V5_ROOT_INODE);
+        assert!(sb.get_inode(1, false).is_err());
+        assert!(sb.get_extended_inode(1, false).is_err());
+
+        // Test adding regular inode
+        let mut inode1 = CachedInodeV5::new(sb.s_blob.clone(), sb.s_meta.clone());
+        inode1.i_ino = 10;
+        inode1.i_nlink = 1;
+        inode1.i_mode = libc::S_IFREG as u32;
+        let inode1_arc = Arc::new(inode1);
+        sb.hash_inode(inode1_arc.clone()).unwrap();
+
+        assert_eq!(sb.get_max_ino(), 10);
+        assert!(sb.get_inode(10, false).is_ok());
+        assert!(sb.get_extended_inode(10, false).is_ok());
+
+        // Test adding hardlink with data (should not replace existing)
+        let mut hardlink = CachedInodeV5::new(sb.s_blob.clone(), sb.s_meta.clone());
+        hardlink.i_ino = 10; // Same inode number
+        hardlink.i_nlink = 2; // Hardlink
+        hardlink.i_mode = libc::S_IFREG as u32;
+        hardlink.i_data = vec![Arc::new(CachedChunkInfoV5::new())]; // Has data
+
+        let hardlink_arc = Arc::new(hardlink);
+        let _result = sb.hash_inode(hardlink_arc.clone()).unwrap();
+
+        // Since original inode has no data, the hardlink with data should replace it
+        let stored_inode = sb.get_inode(10, false).unwrap();
+        assert_eq!(
+            stored_inode
+                .as_any()
+                .downcast_ref::<CachedInodeV5>()
+                .unwrap()
+                .i_data
+                .len(),
+            1
+        );
+
+        // Test root inode
+        assert_eq!(sb.root_ino(), RAFS_V5_ROOT_INODE);
+
+        // Test destroy
+        sb.destroy();
+        assert_eq!(sb.s_inodes.len(), 0);
+    }
+
+    #[test]
+    fn test_cached_superblock_v5_blob_operations() {
+        let md = RafsSuperMeta::default();
+        let sb = CachedSuperBlockV5::new(md, false);
+
+        // Test get_blob_infos with empty blob table
+        let blobs = sb.get_blob_infos();
+        assert!(blobs.is_empty());
+
+        // Note: get_chunk_info() and set_blob_device() both panic with
+        // "not implemented: used by RAFS v6 only" so we can't test them directly
+    }
+
+    #[test]
+    fn test_cached_superblock_v5_hardlink_handling() {
+        let md = RafsSuperMeta::default();
+        let mut sb = CachedSuperBlockV5::new(md, false);
+
+        // Add inode without data
+        let mut inode1 = CachedInodeV5::new(sb.s_blob.clone(), sb.s_meta.clone());
+        inode1.i_ino = 5;
+        inode1.i_nlink = 1;
+        inode1.i_mode = libc::S_IFREG as u32;
+        sb.hash_inode(Arc::new(inode1)).unwrap();
+
+        // Add hardlink with same inode number but no data - should replace
+        let mut hardlink = CachedInodeV5::new(sb.s_blob.clone(), sb.s_meta.clone());
+        hardlink.i_ino = 5;
+        hardlink.i_nlink = 2;
+        hardlink.i_mode = libc::S_IFREG as u32;
+        hardlink.i_data = vec![]; // No data
+
+        sb.hash_inode(Arc::new(hardlink)).unwrap();
+
+        // Should have replaced the original
+        let stored = sb.get_inode(5, false).unwrap();
+        assert_eq!(
+            stored
+                .as_any()
+                .downcast_ref::<CachedInodeV5>()
+                .unwrap()
+                .i_nlink,
+            2
+        );
+    }
+
+    #[test]
+    fn test_from_rafs_v5_chunk_info() {
+        let mut ondisk_chunk = RafsV5ChunkInfo::new();
+        ondisk_chunk.block_id = RafsDigest::from_buf("test".as_bytes(), Algorithm::Blake3);
+        ondisk_chunk.blob_index = 1;
+        ondisk_chunk.index = 42;
+        ondisk_chunk.file_offset = 1024;
+        ondisk_chunk.compressed_offset = 512;
+        ondisk_chunk.uncompressed_offset = 2048;
+        ondisk_chunk.compressed_size = 256;
+        ondisk_chunk.uncompressed_size = 512;
+        ondisk_chunk.flags = BlobChunkFlags::COMPRESSED;
+
+        let cached_chunk = CachedChunkInfoV5::from(&ondisk_chunk);
+
+        assert_eq!(cached_chunk.blob_index(), 1);
+        assert_eq!(cached_chunk.index(), 42);
+        assert_eq!(cached_chunk.file_offset(), 1024);
+        assert_eq!(cached_chunk.compressed_offset(), 512);
+        assert_eq!(cached_chunk.uncompressed_offset(), 2048);
+        assert_eq!(cached_chunk.compressed_size(), 256);
+        assert_eq!(cached_chunk.uncompressed_size(), 512);
+        assert_eq!(cached_chunk.flags(), BlobChunkFlags::COMPRESSED);
+        assert!(cached_chunk.is_compressed());
+    }
+
+    #[test]
+    fn test_cached_inode_v5_accessor_methods() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut inode = CachedInodeV5::new(blob_table, meta);
+
+        // Set test values
+        inode.i_ino = 42;
+        inode.i_size = 8192;
+
+        inode.i_rdev = 0x0801; // Example device number
+        inode.i_projid = 1000;
+        inode.i_parent = 1;
+        inode.i_name = OsString::from("test_file");
+        inode.i_flags = RafsInodeFlags::XATTR;
+        inode.i_digest = RafsDigest::from_buf("test".as_bytes(), Algorithm::Blake3);
+        inode.i_child_idx = 10;
+
+        // Test basic getters
+        assert_eq!(inode.ino(), 42);
+        assert_eq!(inode.size(), 8192);
+        assert_eq!(inode.rdev(), 0x0801);
+        assert_eq!(inode.projid(), 1000);
+        assert_eq!(inode.parent(), 1);
+        assert_eq!(inode.name(), OsString::from("test_file"));
+        assert_eq!(inode.get_name_size(), "test_file".len() as u16);
+        assert_eq!(inode.flags(), RafsInodeFlags::XATTR.bits());
+        assert_eq!(inode.get_digest(), inode.i_digest);
+        assert_eq!(inode.get_child_index().unwrap(), 10);
+
+        // Test as_inode
+        let as_inode = inode.as_inode();
+        assert_eq!(as_inode.ino(), 42);
+    }
+
+    #[test]
+    fn test_cached_inode_v5_edge_cases() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut inode = CachedInodeV5::new(blob_table, meta);
+
+        // Test very large inode number
+        inode.i_ino = u64::MAX;
+        assert_eq!(inode.ino(), u64::MAX);
+
+        // Test edge case file modes
+        inode.i_mode = 0o777 | libc::S_IFREG as u32;
+        assert!(inode.is_reg());
+        assert_eq!(inode.i_mode & 0o777, 0o777);
+
+        // Test empty symlink target (should be invalid but we test getter)
+        inode.i_mode = libc::S_IFLNK as u32;
+        inode.i_target = OsString::new();
+        assert_eq!(inode.get_symlink_size(), 0);
+
+        // Test maximum name length
+        let max_name = "a".repeat(RAFS_MAX_NAME);
+        inode.i_name = OsString::from(max_name.clone());
+        assert_eq!(inode.name(), OsString::from(max_name));
+        assert_eq!(inode.get_name_size(), RAFS_MAX_NAME as u16);
+    }
+
+    #[test]
+    fn test_cached_inode_v5_zero_values() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let inode = CachedInodeV5::new(blob_table, meta);
+
+        // Test all zero/default values
+        assert_eq!(inode.ino(), 0);
+        assert_eq!(inode.size(), 0);
+        assert_eq!(inode.rdev(), 0);
+        assert_eq!(inode.projid(), 0);
+        assert_eq!(inode.parent(), 0);
+        assert_eq!(inode.flags(), 0);
+        assert_eq!(inode.get_name_size(), 0);
+        assert!(!inode.has_xattr());
+        assert!(!inode.is_hardlink());
+
+        // Test get_child operations on empty inode
+        assert_eq!(inode.get_child_count(), 0);
+        assert!(inode.get_child_by_index(0).is_err());
+        assert!(inode.get_child_by_name(OsStr::new("test")).is_err());
+
+        // Test chunk operations on empty inode
+        assert_eq!(inode.i_data.len(), 0);
+        assert!(inode.get_chunk_info(0).is_err());
+    }
+
+    #[test]
+    fn test_cached_chunk_info_v5_boundary_values() {
+        let mut info = CachedChunkInfoV5::new();
+
+        // Test maximum values
+        info.blob_index = u32::MAX;
+        info.index = u32::MAX;
+        info.file_offset = u64::MAX;
+        info.compressed_offset = u64::MAX;
+        info.uncompressed_offset = u64::MAX;
+        info.compressed_size = u32::MAX;
+        info.uncompressed_size = u32::MAX;
+        info.crc32 = u32::MAX;
+
+        assert_eq!(info.blob_index(), u32::MAX);
+        assert_eq!(info.index(), u32::MAX);
+        assert_eq!(info.file_offset(), u64::MAX);
+        assert_eq!(info.compressed_offset(), u64::MAX);
+        assert_eq!(info.uncompressed_offset(), u64::MAX);
+        assert_eq!(info.compressed_size(), u32::MAX);
+        assert_eq!(info.uncompressed_size(), u32::MAX);
+
+        // Test zero values
+        info.blob_index = 0;
+        info.index = 0;
+        info.file_offset = 0;
+        info.compressed_offset = 0;
+        info.uncompressed_offset = 0;
+        info.compressed_size = 0;
+        info.uncompressed_size = 0;
+        info.crc32 = 0;
+
+        assert_eq!(info.blob_index(), 0);
+        assert_eq!(info.index(), 0);
+        assert_eq!(info.file_offset(), 0);
+        assert_eq!(info.compressed_offset(), 0);
+        assert_eq!(info.uncompressed_offset(), 0);
+        assert_eq!(info.compressed_size(), 0);
+        assert_eq!(info.uncompressed_size(), 0);
+        assert_eq!(info.crc32(), 0);
+    }
+
+    #[test]
+    fn test_cached_inode_v5_special_names() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+        let mut inode = CachedInodeV5::new(blob_table, meta);
+
+        // Test special characters in names
+        let special_names = vec![
+            ".",
+            "..",
+            "file with spaces",
+            "file\twith\ttabs",
+            "file\nwith\nnewlines",
+            "file-with-dashes",
+            "file_with_underscores",
+            "file.with.dots",
+            "UPPERCASE_FILE",
+            "MiXeD_cAsE_fIlE",
+            "123456789",
+            "中文文件名", // Chinese characters
+            "файл",       // Cyrillic
+            "🦀🦀🦀",     // Emojis
+        ];
+
+        for name in special_names {
+            inode.i_name = OsString::from(name);
+            assert_eq!(inode.name(), OsString::from(name));
+            assert_eq!(inode.get_name_size(), name.len() as u16);
+        }
+    }
+
+    #[test]
+    fn test_cached_superblock_v5_edge_cases() {
+        let md = RafsSuperMeta::default();
+        let mut sb = CachedSuperBlockV5::new(md, false);
+
+        // Test with validation enabled
+        let md_validated = RafsSuperMeta::default();
+        let sb_validated = CachedSuperBlockV5::new(md_validated, true);
+        assert!(sb_validated.validate_inode);
+
+        // Test maximum inode number
+        let mut inode = CachedInodeV5::new(sb.s_blob.clone(), sb.s_meta.clone());
+        inode.i_ino = u64::MAX;
+        inode.i_nlink = 1;
+        inode.i_mode = libc::S_IFREG as u32;
+        inode.i_name = OsString::from("max_inode");
+
+        sb.hash_inode(Arc::new(inode)).unwrap();
+        assert_eq!(sb.get_max_ino(), u64::MAX);
+
+        // Test getting non-existent inode
+        assert!(sb.get_inode(u64::MAX - 1, false).is_err());
+        assert!(sb.get_extended_inode(u64::MAX - 1, false).is_err());
+
+        // Test blob operations
+        let blob_infos = sb.get_blob_infos();
+        assert!(blob_infos.is_empty());
+
+        let blob_extra_infos = sb.get_blob_extra_infos().unwrap();
+        assert!(blob_extra_infos.is_empty());
+    }
+
+    #[test]
+    fn test_cached_inode_v5_complex_directory_structure() {
+        let meta = Arc::new(RafsSuperMeta::default());
+        let blob_table = Arc::new(RafsV5BlobTable::new());
+
+        // Create a complex directory with many children
+        let mut root_dir = CachedInodeV5::new(blob_table.clone(), meta.clone());
+        root_dir.i_ino = 1;
+        root_dir.i_mode = libc::S_IFDIR as u32;
+        root_dir.i_name = OsString::from("root");
+
+        // Add many children with different names to test sorting
+        let child_names = [
+            "zzz_last",
+            "aaa_first",
+            "mmm_middle",
+            "000_numeric",
+            "ZZZ_upper",
+            "___underscore",
+            "...dots",
+            "111_mixed",
+            "yyy_second_last",
+            "bbb_second",
+        ];
+
+        // Set the correct child count for sorting to trigger
+        root_dir.i_child_cnt = child_names.len() as u32;
+
+        for (i, name) in child_names.iter().enumerate() {
+            let mut child = CachedInodeV5::new(blob_table.clone(), meta.clone());
+            child.i_ino = i as u64 + 2;
+            child.i_name = OsString::from(*name);
+            child.i_mode = if i % 2 == 0 {
+                libc::S_IFREG as u32
+            } else {
+                libc::S_IFDIR as u32
+            };
+            root_dir.add_child(Arc::new(child));
+        }
+
+        // Verify children are sorted by name (after all children are added)
+        assert_eq!(root_dir.i_child.len(), child_names.len());
+        for i in 1..root_dir.i_child.len() {
+            let prev_name = &root_dir.i_child[i - 1].i_name;
+            let curr_name = &root_dir.i_child[i].i_name;
+            assert!(
+                prev_name <= curr_name,
+                "Children not sorted: {:?} > {:?}",
+                prev_name,
+                curr_name
+            );
+        }
+
+        // Test walking all children
+        let mut visited_count = 0;
+        root_dir
+            .walk_children_inodes(0, &mut |_node, _name, _ino, _offset| {
+                visited_count += 1;
+                Ok(RafsInodeWalkAction::Continue)
+            })
+            .unwrap();
+
+        // Should visit ".", "..", and all children
+        assert_eq!(visited_count, 2 + child_names.len());
+
+        // Test collecting descendants
+        let mut descendants = Vec::new();
+        root_dir
+            .collect_descendants_inodes(&mut descendants)
+            .unwrap();
+        // Only regular files with size > 0 are collected, so should be empty
+        assert!(descendants.is_empty());
+    }
 }

service/Cargo.toml

@@ -46,6 +46,9 @@ tokio-uring = "0.4"
 [dev-dependencies]
 vmm-sys-util = "0.12.1"
 
+[target.'cfg(target_os = "linux")'.dev-dependencies]
+procfs = "0.17.0"
+
 [features]
 default = ["fuse-backend-rs/fusedev"]
 virtiofs = [

service/src/singleton.rs

@@ -370,6 +370,7 @@ mod tests {
 
     use super::*;
     use mio::{Poll, Token};
+    use procfs::sys::kernel::Version;
     use vmm_sys_util::tempdir::TempDir;
 
     fn create_service_controller() -> ServiceController {
@@ -416,6 +417,24 @@ mod tests {
             .initialize_fscache_service(None, 1, p.to_str().unwrap(), None)
             .is_err());
 
+        // skip test if user is not root
+        if !nix::unistd::Uid::effective().is_root() {
+            println!("Skip test_initialize_fscache_service, not root");
+            return;
+        }
+
+        // skip test if kernel is older than 5.19
+        if Version::current().unwrap() < Version::from_str("5.19.0").unwrap() {
+            println!("Skip test_initialize_fscache_service, kernel version is older than 5.19");
+            return;
+        }
+
+        // skip test if /dev/cachefiles does not exist
+        if !std::path::Path::new("/dev/cachefiles").exists() {
+            println!("Skip test_initialize_fscache_service, /dev/cachefiles does not exist");
+            return;
+        }
+
         let tmp_dir = TempDir::new().unwrap();
         let dir = tmp_dir.as_path().to_str().unwrap();
         assert!(service_controller

storage/src/backend/connection.rs

@@ -7,14 +7,13 @@ use std::cell::RefCell;
 use std::collections::HashMap;
 use std::io::{Read, Result};
 use std::str::FromStr;
-use std::sync::atomic::{AtomicBool, AtomicI16, AtomicU64, AtomicU8, Ordering};
+use std::sync::atomic::{AtomicBool, AtomicI16, AtomicU64, Ordering};
 use std::sync::Arc;
 use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
 use std::{fmt, thread};
 
 use log::{max_level, Level};
 
-use reqwest::header::{HeaderName, HeaderValue};
 use reqwest::{
     self,
     blocking::{Body, Client, Response},
@@ -23,7 +22,7 @@ use reqwest::{
     Method, StatusCode, Url,
 };
 
-use nydus_api::{HttpProxyConfig, MirrorConfig, OssConfig, ProxyConfig, RegistryConfig, S3Config};
+use nydus_api::{HttpProxyConfig, OssConfig, ProxyConfig, RegistryConfig, S3Config};
 use url::ParseError;
 
 const HEADER_AUTHORIZATION: &str = "Authorization";
@@ -43,8 +42,6 @@ pub enum ConnectionError {
     Format(reqwest::Error),
     Url(String, ParseError),
     Scheme(String),
-    MirrorHost,
-    MirrorPort,
 }
 
 impl fmt::Display for ConnectionError {
@@ -56,8 +53,6 @@ impl fmt::Display for ConnectionError {
             ConnectionError::Format(e) => write!(f, "{}", e),
             ConnectionError::Url(s, e) => write!(f, "failed to parse URL {}, {}", s, e),
             ConnectionError::Scheme(s) => write!(f, "invalid scheme {}", s),
-            ConnectionError::MirrorHost => write!(f, "invalid mirror host"),
-            ConnectionError::MirrorPort => write!(f, "invalid mirror port"),
         }
     }
 }
@@ -69,7 +64,6 @@ type ConnectionResult<T> = std::result::Result<T, ConnectionError>;
 #[derive(Debug, Clone)]
 pub(crate) struct ConnectionConfig {
     pub proxy: ProxyConfig,
-    pub mirrors: Vec<MirrorConfig>,
     pub skip_verify: bool,
     pub timeout: u32,
     pub connect_timeout: u32,
@@ -80,7 +74,6 @@ impl Default for ConnectionConfig {
     fn default() -> Self {
         Self {
             proxy: ProxyConfig::default(),
-            mirrors: Vec::<MirrorConfig>::new(),
             skip_verify: false,
             timeout: 5,
             connect_timeout: 5,
@@ -93,7 +86,6 @@ impl From<OssConfig> for ConnectionConfig {
     fn from(c: OssConfig) -> ConnectionConfig {
         ConnectionConfig {
             proxy: c.proxy,
-            mirrors: c.mirrors,
             skip_verify: c.skip_verify,
             timeout: c.timeout,
             connect_timeout: c.connect_timeout,
@@ -106,7 +98,6 @@ impl From<S3Config> for ConnectionConfig {
     fn from(c: S3Config) -> ConnectionConfig {
         ConnectionConfig {
             proxy: c.proxy,
-            mirrors: c.mirrors,
             skip_verify: c.skip_verify,
             timeout: c.timeout,
             connect_timeout: c.connect_timeout,
@@ -119,7 +110,6 @@ impl From<RegistryConfig> for ConnectionConfig {
     fn from(c: RegistryConfig) -> ConnectionConfig {
         ConnectionConfig {
             proxy: c.proxy,
-            mirrors: c.mirrors,
             skip_verify: c.skip_verify,
             timeout: c.timeout,
             connect_timeout: c.connect_timeout,
@@ -132,7 +122,6 @@ impl From<HttpProxyConfig> for ConnectionConfig {
     fn from(c: HttpProxyConfig) -> ConnectionConfig {
         ConnectionConfig {
             proxy: c.proxy,
-            mirrors: c.mirrors,
             skip_verify: c.skip_verify,
             timeout: c.timeout,
             connect_timeout: c.connect_timeout,
@@ -264,45 +253,11 @@ pub(crate) fn respond(resp: Response, catch_status: bool) -> ConnectionResult<Re
 pub(crate) struct Connection {
     client: Client,
     proxy: Option<Arc<Proxy>>,
-    pub mirrors: Vec<Arc<Mirror>>,
     pub shutdown: AtomicBool,
     /// Timestamp of connection's last active request, represents as duration since UNIX_EPOCH in seconds.
     last_active: Arc<AtomicU64>,
 }
 
-#[derive(Debug)]
-pub(crate) struct Mirror {
-    /// Information for mirror from configuration file.
-    pub config: MirrorConfig,
-    /// Mirror status, it will be set to false by atomic operation when mirror is not work.
-    status: AtomicBool,
-    /// Failed times requesting mirror, the status will be marked as false when failed_times = failure_limit.
-    failed_times: AtomicU8,
-    /// Failure count for which mirror is considered unavailable.
-    failure_limit: u8,
-}
-
-impl Mirror {
-    /// Convert original URL to mirror URL.
-    fn mirror_url(&self, url: &str) -> ConnectionResult<Url> {
-        let mirror_host = Url::parse(&self.config.host)
-            .map_err(|e| ConnectionError::Url(self.config.host.clone(), e))?;
-        let mut current_url =
-            Url::parse(url).map_err(|e| ConnectionError::Url(url.to_string(), e))?;
-
-        current_url
-            .set_scheme(mirror_host.scheme())
-            .map_err(|_| ConnectionError::Scheme(mirror_host.scheme().to_string()))?;
-        current_url
-            .set_host(mirror_host.host_str())
-            .map_err(|_| ConnectionError::MirrorHost)?;
-        current_url
-            .set_port(mirror_host.port())
-            .map_err(|_| ConnectionError::MirrorPort)?;
-        Ok(current_url)
-    }
-}
-
 impl Connection {
     /// Create a new connection according to the configuration.
     pub fn new(config: &ConnectionConfig) -> Result<Arc<Connection>> {
@@ -330,22 +285,9 @@ impl Connection {
             None
         };
 
-        let mut mirrors = Vec::new();
-        for mirror_config in config.mirrors.iter() {
-            if !mirror_config.host.is_empty() {
-                mirrors.push(Arc::new(Mirror {
-                    config: mirror_config.clone(),
-                    status: AtomicBool::from(true),
-                    failed_times: AtomicU8::from(0),
-                    failure_limit: mirror_config.failure_limit,
-                }));
-            }
-        }
-
         let connection = Arc::new(Connection {
             client,
             proxy,
-            mirrors,
             shutdown: AtomicBool::new(false),
             last_active: Arc::new(AtomicU64::new(
                 SystemTime::now()
@@ -358,9 +300,6 @@ impl Connection {
         // Start proxy's health checking thread.
         connection.start_proxy_health_thread(config.connect_timeout as u64);
 
-        // Start mirrors' health checking thread.
-        connection.start_mirrors_health_thread(config.timeout as u64);
-
         Ok(connection)
     }
 
@@ -417,72 +356,6 @@ impl Connection {
         }
     }
 
-    fn start_mirrors_health_thread(&self, timeout: u64) {
-        for mirror in self.mirrors.iter() {
-            let mirror_cloned = mirror.clone();
-            let last_active = Arc::clone(&self.last_active);
-
-            // Spawn thread to update the health status of mirror server.
-            thread::spawn(move || {
-                let mirror_health_url = if mirror_cloned.config.ping_url.is_empty() {
-                    format!("{}/v2", mirror_cloned.config.host)
-                } else {
-                    mirror_cloned.config.ping_url.clone()
-                };
-                info!(
-                    "[mirror] start health check, ping url: {}",
-                    mirror_health_url
-                );
-
-                let client = Client::new();
-                loop {
-                    let elapsed = SystemTime::now()
-                        .duration_since(UNIX_EPOCH)
-                        .unwrap()
-                        .as_secs()
-                        - last_active.load(Ordering::Relaxed);
-                    // If the connection is not active for a set time, skip mirror health check.
-                    if elapsed <= mirror_cloned.config.health_check_pause_elapsed {
-                        // Try to recover the mirror server when it is unavailable.
-                        if !mirror_cloned.status.load(Ordering::Relaxed) {
-                            info!(
-                                "[mirror] server unhealthy, try to recover: {}",
-                                mirror_cloned.config.host
-                            );
-
-                            let _ = client
-                                .get(mirror_health_url.as_str())
-                                .timeout(Duration::from_secs(timeout as u64))
-                                .send()
-                                .map(|resp| {
-                                    // If the response status is less than StatusCode::INTERNAL_SERVER_ERROR,
-                                    // the mirror server is recovered.
-                                    if resp.status() < StatusCode::INTERNAL_SERVER_ERROR {
-                                        info!(
-                                            "[mirror] server recovered: {}",
-                                            mirror_cloned.config.host
-                                        );
-                                        mirror_cloned.failed_times.store(0, Ordering::Relaxed);
-                                        mirror_cloned.status.store(true, Ordering::Relaxed);
-                                    }
-                                })
-                                .map_err(|e| {
-                                    warn!(
-                                        "[mirror] failed to recover server: {}, {}",
-                                        mirror_cloned.config.host, e
-                                    );
-                                });
-                        }
-                    }
-
-                    thread::sleep(Duration::from_secs(
-                        mirror_cloned.config.health_check_interval,
-                    ));
-                }
-            });
-        }
-    }
-
     /// Shutdown the connection.
     pub fn shutdown(&self) {
         self.shutdown.store(true, Ordering::Release);
@@ -562,69 +435,6 @@ impl Connection {
             }
         }
 
-        let mut mirror_enabled = false;
-        if !self.mirrors.is_empty() {
-            mirror_enabled = true;
-            for mirror in self.mirrors.iter() {
-                if mirror.status.load(Ordering::Relaxed) {
-                    let data_cloned = data.as_ref().cloned();
-
-                    for (key, value) in mirror.config.headers.iter() {
-                        headers.insert(
-                            HeaderName::from_str(key).unwrap(),
-                            HeaderValue::from_str(value).unwrap(),
-                        );
-                    }
-
-                    let current_url = mirror.mirror_url(url)?;
-                    debug!("[mirror] replace to: {}", current_url);
-
-                    let result = self.call_inner(
-                        &self.client,
-                        method.clone(),
-                        current_url.as_str(),
-                        &query,
-                        data_cloned,
-                        headers,
-                        catch_status,
-                        false,
-                    );
-
-                    match result {
-                        Ok(resp) => {
-                            // If the response status >= INTERNAL_SERVER_ERROR, move to the next mirror server.
-                            if resp.status() < StatusCode::INTERNAL_SERVER_ERROR {
-                                return Ok(resp);
-                            }
-                        }
-                        Err(err) => {
-                            warn!(
-                                "[mirror] request failed, server: {:?}, {:?}",
-                                mirror.config.host, err
-                            );
-                            mirror.failed_times.fetch_add(1, Ordering::Relaxed);
-
-                            if mirror.failed_times.load(Ordering::Relaxed) >= mirror.failure_limit {
-                                warn!(
-                                    "[mirror] exceed failure limit {}, server disabled: {:?}",
-                                    mirror.failure_limit, mirror
-                                );
-                                mirror.status.store(false, Ordering::Relaxed);
-                            }
-                        }
-                    }
-                }
-                // Remove mirror-related headers to avoid sending them to the next mirror server and original registry.
-                for (key, _) in mirror.config.headers.iter() {
-                    headers.remove(HeaderName::from_str(key).unwrap());
-                }
-            }
-        }
-
-        if mirror_enabled {
-            warn!("[mirror] request all servers failed, fallback to original server.");
-        }
-
         self.call_inner(
             &self.client,
             method,
@@ -788,6 +598,5 @@ mod tests {
         assert!(config.proxy.fallback);
         assert_eq!(config.proxy.ping_url, "");
         assert_eq!(config.proxy.url, "");
-        assert!(config.mirrors.is_empty());
     }
 }

storage/src/backend/registry.rs

@@ -31,11 +31,6 @@ const REGISTRY_CLIENT_ID: &str = "nydus-registry-client";
 const HEADER_AUTHORIZATION: &str = "Authorization";
 const HEADER_WWW_AUTHENTICATE: &str = "www-authenticate";
 
-const REDIRECTED_STATUS_CODE: [StatusCode; 2] = [
-    StatusCode::MOVED_PERMANENTLY,
-    StatusCode::TEMPORARY_REDIRECT,
-];
-
 const REGISTRY_DEFAULT_TOKEN_EXPIRATION: u64 = 10 * 60; // in seconds
 
 /// Error codes related to registry storage backend operations.
@@ -437,17 +432,21 @@ impl RegistryState {
                 Some(Auth::Basic(BasicAuth { realm }))
             }
             "Bearer" => {
-                if !paras.contains_key("realm")
-                    || !paras.contains_key("service")
-                    || !paras.contains_key("scope")
-                {
+                if !paras.contains_key("realm") || !paras.contains_key("service") {
                     return None;
                 }
 
+                let scope = if let Some(scope) = paras.get("scope") {
+                    (*scope).to_string()
+                } else {
+                    debug!("no scope specified for token auth challenge");
+                    String::new()
+                };
+
                 Some(Auth::Bearer(BearerAuth {
                     realm: (*paras.get("realm").unwrap()).to_string(),
                     service: (*paras.get("service").unwrap()).to_string(),
-                    scope: (*paras.get("scope").unwrap()).to_string(),
+                    scope,
                 }))
             }
             _ => None,
@@ -724,9 +723,11 @@ impl RegistryReader {
                 }
             };
             let status = resp.status();
+            let need_redirect =
+                status >= StatusCode::MULTIPLE_CHOICES && status < StatusCode::BAD_REQUEST;
 
             // Handle redirect request and cache redirect url
-            if REDIRECTED_STATUS_CODE.contains(&status) {
+            if need_redirect {
                 if let Some(location) = resp.headers().get("location") {
                     let location = location.to_str().unwrap();
                     let mut location = Url::parse(location)
@@ -864,12 +865,6 @@ impl Registry {
         let id = id.ok_or_else(|| einval!("Registry backend requires blob_id"))?;
         let con_config: ConnectionConfig = config.clone().into();
 
-        if !config.proxy.url.is_empty() && !config.mirrors.is_empty() {
-            return Err(einval!(
-                "connection: proxy and mirrors cannot be configured at the same time."
-            ));
-        }
-
         let retry_limit = con_config.retry_limit;
         let connection = Connection::new(&con_config)?;
         let auth = trim(config.auth.clone());
@@ -1115,12 +1110,25 @@ mod tests {
             _ => panic!("failed to parse `Bearer` authentication header"),
         }
 
+        // No scope is accetpable
+        let str = "Bearer realm=\"https://auth.my-registry.com/token\",service=\"my-registry.com\"";
+        let header = HeaderValue::from_str(str).unwrap();
+        let auth = RegistryState::parse_auth(&header).unwrap();
+        match auth {
+            Auth::Bearer(auth) => {
+                assert_eq!(&auth.realm, "https://auth.my-registry.com/token");
+                assert_eq!(&auth.service, "my-registry.com");
+                assert_eq!(&auth.scope, "");
+            }
+            _ => panic!("failed to parse `Bearer` authentication header without scope"),
+        }
+
         let str = "Basic realm=\"https://auth.my-registry.com/token\"";
         let header = HeaderValue::from_str(str).unwrap();
         let auth = RegistryState::parse_auth(&header).unwrap();
         match auth {
             Auth::Basic(auth) => assert_eq!(&auth.realm, "https://auth.my-registry.com/token"),
-            _ => panic!("failed to parse `Bearer` authentication header"),
+            _ => panic!("failed to parse `Basic` authentication header"),
         }
 
         let str = "Base realm=\"https://auth.my-registry.com/token\"";