rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5 heartwood59d1548e5afd285fe0a051b3ab9ed127937df77d
{
"request": "trigger",
"version": 1,
"event_type": "patch",
"repository": {
"id": "rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5",
"name": "heartwood",
"description": "Radicle Heartwood Protocol & Stack",
"private": false,
"default_branch": "master",
"delegates": [
"did:key:z6MksFqXN3Yhqk8pTJdUGLwATkRfQvwZXPqR2qMEhbS9wzpT",
"did:key:z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM",
"did:key:z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz"
]
},
"action": "Created",
"patch": {
"id": "875cc60381632048bfe6bee8ffa8b49f63921fec",
"author": {
"id": "did:key:z6MkkfM3tPXNPrPevKr3uSiQtHPuwnNhu2yUVjgd2jXVsVz5",
"alias": "sebastinez"
},
"title": "httpd: Fetch projects after starting seeding it",
"state": {
"status": "archived",
"conflicts": []
},
"before": "0908c65f504ec56bcb368b3a020ddee5f251eb56",
"after": "59d1548e5afd285fe0a051b3ab9ed127937df77d",
"commits": [
"59d1548e5afd285fe0a051b3ab9ed127937df77d",
"936c69c9df5bfd5e1354cf8acd7fb72af54a95ba"
],
"target": "7e1cb406bb935c3de008d2171e8961f57b4336ea",
"labels": [],
"assignees": [],
"revisions": [
{
"id": "875cc60381632048bfe6bee8ffa8b49f63921fec",
"author": {
"id": "did:key:z6MkkfM3tPXNPrPevKr3uSiQtHPuwnNhu2yUVjgd2jXVsVz5",
"alias": "sebastinez"
},
"description": "Implements the fetching behavior from `radicle_cli::commands::rad_sync::fetch`\ninto the `PUT /node/policies/repos/:rid` endpoint.\n\nThis allows users to send a request for adding a repo to their seeding\ntable and the repo will be fetched from where ever possible, allowing\nusers to interact with it quickly and contribute to its seeding.\n\nAlso moving `RepoSync` from `radicle-cli` to `radicle::node`\n\nThis to allow `radicle-httpd` using `RepoSync` without importing it from\n`radicle-cli`.",
"base": "0908c65f504ec56bcb368b3a020ddee5f251eb56",
"oid": "6d6618a907ae372fb6edb7ca333bc045f9521f02",
"timestamp": 1708623235
},
{
"id": "7cba2d27cdc2a92699c7ae86c96f9bd7961711a3",
"author": {
"id": "did:key:z6MkkfM3tPXNPrPevKr3uSiQtHPuwnNhu2yUVjgd2jXVsVz5",
"alias": "sebastinez"
},
"description": "Revert references change to `rad_sync::fetch`",
"base": "0908c65f504ec56bcb368b3a020ddee5f251eb56",
"oid": "59d1548e5afd285fe0a051b3ab9ed127937df77d",
"timestamp": 1708685180
}
]
}
}
{
"response": "triggered",
"run_id": {
"id": "65af05c4-7110-492d-91c2-17453c52c90f"
},
"info_url": "https://cci.rad.levitte.org//65af05c4-7110-492d-91c2-17453c52c90f.html"
}
Started at: 2026-09-08 22:08:45.347443+02:00
Commands:
$ rad clone rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5 .
✓ Creating checkout in ./...
✓ Remote cloudhead@z6MksFqXN3Yhqk8pTJdUGLwATkRfQvwZXPqR2qMEhbS9wzpT added
✓ Remote-tracking branch cloudhead@z6MksFqXN3Yhqk8pTJdUGLwATkRfQvwZXPqR2qMEhbS9wzpT/master created for z6MksFqXN3Yhqk8pTJdUGLwATkRfQvwZXPqR2qMEhbS9wzpT
✓ Remote fintohaps@z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM added
✓ Remote-tracking branch fintohaps@z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM/master created for z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM
✓ Remote lorenz@z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz added
✓ Remote-tracking branch lorenz@z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz/master created for z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz
✓ Repository successfully cloned under /opt/radcis/ci.rad.levitte.org/cci/state/65af05c4-7110-492d-91c2-17453c52c90f/w/
╭────────────────────────────────────╮
│ heartwood │
│ Radicle Heartwood Protocol & Stack │
│ 189 issues · 39 patches │
╰────────────────────────────────────╯
Run `cd ./.` to go to the repository directory.
Exit code: 0
$ rad patch checkout 875cc60381632048bfe6bee8ffa8b49f63921fec
✓ Switched to branch patch/875cc60 at revision 7cba2d2
✓ Branch patch/875cc60 setup to track rad/patches/875cc60381632048bfe6bee8ffa8b49f63921fec
Exit code: 0
$ git config advice.detachedHead false
Exit code: 0
$ git checkout 59d1548e5afd285fe0a051b3ab9ed127937df77d
HEAD is now at 59d1548e httpd: Fetch projects after starting seeding it
Exit code: 0
$ rad patch show 875cc60381632048bfe6bee8ffa8b49f63921fec -p
╭─────────────────────────────────────────────────────────────────────────────────╮
│ Title httpd: Fetch projects after starting seeding it │
│ Patch 875cc60381632048bfe6bee8ffa8b49f63921fec │
│ Author sebastinez z6MkkfM…jXVsVz5 │
│ Head 59d1548e5afd285fe0a051b3ab9ed127937df77d │
│ Base 0908c65f504ec56bcb368b3a020ddee5f251eb56 │
│ Target master │
│ Branches patch/875cc60 │
│ Commits ahead 2, behind 1610 │
│ Status archived │
│ │
│ Implements the fetching behavior from `radicle_cli::commands::rad_sync::fetch` │
│ into the `PUT /node/policies/repos/:rid` endpoint. │
│ │
│ This allows users to send a request for adding a repo to their seeding │
│ table and the repo will be fetched from where ever possible, allowing │
│ users to interact with it quickly and contribute to its seeding. │
│ │
│ Also moving `RepoSync` from `radicle-cli` to `radicle::node` │
│ │
│ This to allow `radicle-httpd` using `RepoSync` without importing it from │
│ `radicle-cli`. │
├─────────────────────────────────────────────────────────────────────────────────┤
│ 59d1548 httpd: Fetch projects after starting seeding it │
│ 936c69c radicle: Moving `RepoSync` from `radicle-cli` to `radicle::node` │
├─────────────────────────────────────────────────────────────────────────────────┤
│ ● Revision 875cc60 @ 0908c65..6d6618a by sebastinez z6MkkfM…jXVsVz5 2 years ago │
│ ↑ Revision 7cba2d2 @ 0908c65..59d1548 by sebastinez z6MkkfM…jXVsVz5 2 years ago │
╰─────────────────────────────────────────────────────────────────────────────────╯
commit 59d1548e5afd285fe0a051b3ab9ed127937df77d
Author: Sebastian Martinez <me@sebastinez.dev>
Date: Thu Feb 22 14:32:50 2024 -0300
httpd: Fetch projects after starting seeding it
Implements the fetching behavior from `radicle_cli::commands::rad_sync::fetch`
into the `PUT /node/policies/repos/:rid` endpoint.
This allows users to send a request for adding a repo to their seeding
table and the repo will be fetched from where ever possible, allowing
users to interact with it quickly and contribute to its seeding.
diff --git a/radicle-cli/src/commands/sync.rs b/radicle-cli/src/commands/sync.rs
index b9567c4885..32372ae64d 100644
--- a/radicle-cli/src/commands/sync.rs
+++ b/radicle-cli/src/commands/sync.rs
@@ -500,14 +500,14 @@ pub fn fetch(
// Fetch from connected seeds.
let connected = connected
- .iter()
+ .into_iter()
.filter(|c| !results.contains(&c.nid))
- .map(|c| &c.nid)
+ .map(|c| c.nid)
.take(replicas)
.collect::<Vec<_>>();
for nid in connected {
- let result = fetch_from(rid, nid, timeout, node)?;
- results.push(*nid, result);
+ let result = fetch_from(rid, &nid, timeout, node)?;
+ results.push(nid, result);
}
// Try to connect to disconnected seeds and fetch from them.
diff --git a/radicle-httpd/src/api.rs b/radicle-httpd/src/api.rs
index 75fb62183d..6246638391 100644
--- a/radicle-httpd/src/api.rs
+++ b/radicle-httpd/src/api.rs
@@ -19,7 +19,7 @@ use radicle::cob::patch;
use radicle::identity::{DocAt, RepoId};
use radicle::node::policy::Scope;
use radicle::node::routing::Store;
-use radicle::node::{Handle, NodeId};
+use radicle::node::Handle;
use radicle::storage::{ReadRepository, ReadStorage};
use radicle::{Node, Profile};
@@ -156,8 +156,6 @@ pub struct CobsQuery<T> {
#[derive(Serialize, Deserialize, Clone)]
#[serde(rename_all = "camelCase")]
pub struct PoliciesQuery {
- /// The NID from which to fetch from after tracking a repo.
- pub from: Option<NodeId>,
pub scope: Option<Scope>,
}
diff --git a/radicle-httpd/src/api/v1/node.rs b/radicle-httpd/src/api/v1/node.rs
index 0181868de9..fd976be2db 100644
--- a/radicle-httpd/src/api/v1/node.rs
+++ b/radicle-httpd/src/api/v1/node.rs
@@ -8,7 +8,7 @@ use serde_json::json;
use radicle::identity::RepoId;
use radicle::node::routing::Store;
-use radicle::node::{policy, AliasStore, Handle, NodeId, DEFAULT_TIMEOUT};
+use radicle::node::{self, policy, AliasStore, FetchResults, Handle, NodeId, DEFAULT_TIMEOUT};
use radicle::Node;
use crate::api::error::Error;
@@ -109,17 +109,79 @@ async fn node_policies_seed_handler(
Query(qs): Query<PoliciesQuery>,
) -> impl IntoResponse {
api::auth::validate(&ctx, &token).await?;
- let mut node = Node::new(ctx.profile.socket());
- node.seed(project, qs.scope.unwrap_or_default())?;
-
- if let Some(from) = qs.from {
- let results = node.fetch(project, from, DEFAULT_TIMEOUT)?;
+ let profile = &ctx.profile;
+ let mut node = Node::new(profile.socket());
+ let updated = node.seed(project, qs.scope.unwrap_or_default())?;
+ let outcome = if updated { "updated" } else { "exists" };
+
+ let local = ctx.profile.id();
+ let seeds = node.seeds(project)?;
+ let settings = node::seed::RepoSync::default().with_profile(profile);
+ let replicas = settings
+ .replicas
+ .min(seeds.iter().filter(|s| s.nid != *local).count());
+ let sessions = node.sessions()?;
+ let mut results = FetchResults::default();
+ let (connected, mut disconnected) = seeds.partition();
+
+ // Fetch from specified seeds, plus our preferred seeds.
+ for nid in &settings.seeds {
+ if !sessions.iter().any(|s| s.nid == *nid) {
+ // Node isn't connected, skipping it.
+ continue;
+ }
+ let result = node.fetch(project, *nid, DEFAULT_TIMEOUT)?;
+ results.push(*nid, result);
+ }
+ if results.success().count() >= replicas {
return Ok::<_, Error>((
StatusCode::OK,
- Json(json!({ "success": true, "results": results })),
+ Json(json!({ "success": true, "outcome": outcome, "results": results })),
));
}
- Ok::<_, Error>((StatusCode::OK, Json(json!({ "success": true }))))
+
+ // Fetch from connected seeds.
+ let connected = connected
+ .into_iter()
+ .filter(|c| !results.contains(&c.nid))
+ .map(|c| c.nid)
+ .take(replicas)
+ .collect::<Vec<_>>();
+ for nid in connected {
+ let result = node.fetch(project, nid, DEFAULT_TIMEOUT)?;
+ results.push(nid, result);
+ }
+
+ // Try to connect to disconnected seeds and fetch from them.
+ while results.success().count() < replicas {
+ let Some(seed) = disconnected.pop() else {
+ break;
+ };
+ if seed.nid == *local {
+ // Skip our own node.
+ continue;
+ }
+ for addr in seed.addrs.into_iter().map(|ka| ka.addr) {
+ if let node::ConnectResult::Connected = node.connect(
+ seed.nid,
+ addr,
+ node::ConnectOptions {
+ persistent: false,
+ timeout: DEFAULT_TIMEOUT,
+ },
+ )? {
+ let result = node.fetch(project, seed.nid, DEFAULT_TIMEOUT)?;
+ results.push(seed.nid, result);
+ } else {
+ continue;
+ }
+ }
+ }
+
+ Ok::<_, Error>((
+ StatusCode::OK,
+ Json(json!({ "success": true, "outcome": outcome, "results": results })),
+ ))
}
/// Unseed a repo.
diff --git a/radicle/src/node.rs b/radicle/src/node.rs
index 08e6442e20..d57ca7ed51 100644
--- a/radicle/src/node.rs
+++ b/radicle/src/node.rs
@@ -694,7 +694,7 @@ impl<S: ToString> From<Result<(Vec<RefUpdate>, HashSet<NodeId>), S>> for FetchRe
}
/// Holds multiple fetch results.
-#[derive(Debug, Default)]
+#[derive(Debug, Default, Serialize)]
pub struct FetchResults(Vec<(NodeId, FetchResult)>);
impl FetchResults {
commit 936c69c9df5bfd5e1354cf8acd7fb72af54a95ba
Author: Sebastian Martinez <me@sebastinez.dev>
Date: Thu Feb 22 14:31:40 2024 -0300
radicle: Moving `RepoSync` from `radicle-cli` to `radicle::node`
This to allow `radicle-httpd` using `RepoSync` without importing it from
`radicle-cli`.
diff --git a/radicle-cli/src/commands/clone.rs b/radicle-cli/src/commands/clone.rs
index f46fe015af..debe2a50df 100644
--- a/radicle-cli/src/commands/clone.rs
+++ b/radicle-cli/src/commands/clone.rs
@@ -61,7 +61,7 @@ pub struct Options {
/// The seeding scope of the repository.
scope: Scope,
/// Sync settings.
- sync: sync::RepoSync,
+ sync: node::seed::RepoSync,
/// Fetch timeout.
timeout: time::Duration,
}
@@ -73,7 +73,7 @@ impl Args for Options {
let mut parser = lexopt::Parser::from_args(args);
let mut id: Option<RepoId> = None;
let mut scope = Scope::All;
- let mut sync = sync::RepoSync::default();
+ let mut sync = node::seed::RepoSync::default();
let mut timeout = time::Duration::from_secs(9);
let mut directory = None;
@@ -235,7 +235,7 @@ pub fn clone<G: Signer>(
id: RepoId,
directory: Option<PathBuf>,
scope: Scope,
- settings: sync::RepoSync,
+ settings: node::seed::RepoSync,
timeout: time::Duration,
node: &mut Node,
signer: &G,
diff --git a/radicle-cli/src/commands/remote/add.rs b/radicle-cli/src/commands/remote/add.rs
index 22112edbd4..1bb3d37cd4 100644
--- a/radicle-cli/src/commands/remote/add.rs
+++ b/radicle-cli/src/commands/remote/add.rs
@@ -3,6 +3,7 @@ use std::time;
use radicle::git;
use radicle::git::RefString;
+use radicle::node;
use radicle::prelude::*;
use radicle::Profile;
use radicle_crypto::PublicKey;
@@ -31,7 +32,7 @@ pub fn run(
follow::follow(*nid, alias, &mut node, profile)?;
sync::fetch(
rid,
- sync::RepoSync::default().with_profile(profile),
+ node::seed::RepoSync::default().with_profile(profile),
time::Duration::from_secs(9),
&mut node,
)?;
diff --git a/radicle-cli/src/commands/seed.rs b/radicle-cli/src/commands/seed.rs
index 397ac16ad1..7536000d00 100644
--- a/radicle-cli/src/commands/seed.rs
+++ b/radicle-cli/src/commands/seed.rs
@@ -3,6 +3,7 @@ use std::time;
use anyhow::anyhow;
+use radicle::node;
use radicle::node::policy;
use radicle::node::policy::Scope;
use radicle::node::Handle;
@@ -137,7 +138,7 @@ pub fn run(options: Options, ctx: impl term::Context) -> anyhow::Result<()> {
if fetch && node.is_running() {
sync::fetch(
rid,
- sync::RepoSync::default().with_profile(&profile),
+ node::seed::RepoSync::default().with_profile(&profile),
time::Duration::from_secs(6),
&mut node,
)?;
diff --git a/radicle-cli/src/commands/sync.rs b/radicle-cli/src/commands/sync.rs
index bd1ea0cc63..b9567c4885 100644
--- a/radicle-cli/src/commands/sync.rs
+++ b/radicle-cli/src/commands/sync.rs
@@ -8,9 +8,8 @@ use std::time;
use anyhow::{anyhow, Context as _};
use radicle::node;
-use radicle::node::AliasStore;
-use radicle::node::Seed;
-use radicle::node::{FetchResult, FetchResults, Handle as _, Node, SyncStatus};
+use radicle::node::seed::RepoSync;
+use radicle::node::{AliasStore, FetchResult, FetchResults, Handle as _, Node, Seed, SyncStatus};
use radicle::prelude::{NodeId, Profile, RepoId};
use radicle::storage::{ReadRepository, ReadStorage};
use radicle_term::Element;
@@ -114,52 +113,6 @@ impl Default for SyncMode {
}
}
-/// Repository sync settings.
-#[derive(Debug, Clone, PartialEq, Eq)]
-pub struct RepoSync {
- /// Sync with at least N replicas.
- pub replicas: usize,
- /// Sync with the given list of seeds.
- pub seeds: BTreeSet<NodeId>,
-}
-
-impl RepoSync {
- pub fn from_seeds(seeds: impl IntoIterator<Item = NodeId>) -> Self {
- let seeds = BTreeSet::from_iter(seeds);
- Self {
- replicas: seeds.len(),
- seeds,
- }
- }
-
- /// Use profile to populate sync settings, by adding preferred seeds if no seeds are specified,
- /// and removing the local node from the set.
- pub fn with_profile(mut self, profile: &Profile) -> Self {
- // If no seeds were specified, add up to `replica` seeds from the preferred seeds.
- if self.seeds.is_empty() {
- self.seeds = profile
- .config
- .preferred_seeds
- .iter()
- .map(|p| p.id)
- .take(self.replicas)
- .collect();
- }
- // Remove our local node from the seed set just in case it was added by mistake.
- self.seeds.remove(profile.id());
- self
- }
-}
-
-impl Default for RepoSync {
- fn default() -> Self {
- Self {
- replicas: 3,
- seeds: BTreeSet::new(),
- }
- }
-}
-
#[derive(Debug, Default, PartialEq, Eq, Clone)]
pub enum SyncDirection {
Fetch,
@@ -547,14 +500,14 @@ pub fn fetch(
// Fetch from connected seeds.
let connected = connected
- .into_iter()
+ .iter()
.filter(|c| !results.contains(&c.nid))
- .map(|c| c.nid)
+ .map(|c| &c.nid)
.take(replicas)
.collect::<Vec<_>>();
for nid in connected {
- let result = fetch_from(rid, &nid, timeout, node)?;
- results.push(nid, result);
+ let result = fetch_from(rid, nid, timeout, node)?;
+ results.push(*nid, result);
}
// Try to connect to disconnected seeds and fetch from them.
diff --git a/radicle/src/node/seed.rs b/radicle/src/node/seed.rs
index 3408e19675..8eeefef9db 100644
--- a/radicle/src/node/seed.rs
+++ b/radicle/src/node/seed.rs
@@ -1,12 +1,14 @@
pub mod store;
+use std::collections::BTreeSet;
+
pub use store::{Error, Store};
use localtime::LocalTime;
-use crate::git;
use crate::node::KnownAddress;
use crate::prelude::NodeId;
use crate::storage::{refs::RefsAt, ReadRepository, RemoteId};
+use crate::{git, Profile};
/// Holds an oid and timestamp.
#[derive(Debug, Copy, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
@@ -60,3 +62,49 @@ pub struct SyncedSeed {
/// Sync information for a given repo.
pub synced_at: SyncedAt,
}
+
+/// Repository sync settings.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct RepoSync {
+ /// Sync with at least N replicas.
+ pub replicas: usize,
+ /// Sync with the given list of seeds.
+ pub seeds: BTreeSet<NodeId>,
+}
+
+impl RepoSync {
+ pub fn from_seeds(seeds: impl IntoIterator<Item = NodeId>) -> Self {
+ let seeds = BTreeSet::from_iter(seeds);
+ Self {
+ replicas: seeds.len(),
+ seeds,
+ }
+ }
+
+ /// Use profile to populate sync settings, by adding preferred seeds if no seeds are specified,
+ /// and removing the local node from the set.
+ pub fn with_profile(mut self, profile: &Profile) -> Self {
+ // If no seeds were specified, add up to `replica` seeds from the preferred seeds.
+ if self.seeds.is_empty() {
+ self.seeds = profile
+ .config
+ .preferred_seeds
+ .iter()
+ .map(|p| p.id)
+ .take(self.replicas)
+ .collect();
+ }
+ // Remove our local node from the seed set just in case it was added by mistake.
+ self.seeds.remove(profile.id());
+ self
+ }
+}
+
+impl Default for RepoSync {
+ fn default() -> Self {
+ Self {
+ replicas: 3,
+ seeds: BTreeSet::new(),
+ }
+ }
+}
Exit code: 0
shell: 'cargo --version rustc --version cargo fmt --check cargo clippy --all-targets --workspace -- --deny clippy::all cargo build --all-targets --workspace cargo doc --workspace cargo test --workspace --no-fail-fast '
Commands:
$ podman run --name 65af05c4-7110-492d-91c2-17453c52c90f -v /opt/radcis/ci.rad.levitte.org/cci/state/65af05c4-7110-492d-91c2-17453c52c90f/s:/65af05c4-7110-492d-91c2-17453c52c90f/s:ro -v /opt/radcis/ci.rad.levitte.org/cci/state/65af05c4-7110-492d-91c2-17453c52c90f/w:/65af05c4-7110-492d-91c2-17453c52c90f/w -w /65af05c4-7110-492d-91c2-17453c52c90f/w -v /opt/radcis/ci.rad.levitte.org/.radicle:/${id}/.radicle:ro -e RAD_HOME=/${id}/.radicle rust:trixie bash /65af05c4-7110-492d-91c2-17453c52c90f/s/script.sh
+ cargo --version
info: syncing channel updates for '1.74-x86_64-unknown-linux-gnu'
info: latest update on 2023-12-07, rust version 1.74.1 (a28077b28 2023-12-04)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
info: installing component 'rustc'
cargo 1.74.1 (ecb9851af 2023-10-18)
+ rustc --version
rustc 1.74.1 (a28077b28 2023-12-04)
+ cargo fmt --check
error: 'cargo-fmt' is not installed for the toolchain '1.74-x86_64-unknown-linux-gnu'.
To install, run `rustup component add --toolchain 1.74-x86_64-unknown-linux-gnu rustfmt`
Exit code: 1
{
"response": "finished",
"result": "failure"
}