Skip to content

Add project clone retries functionality #1613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dkwon17 wants to merge 2 commits into
base: main
Choose a base branch
from
+50 −8
Open

Add project clone retries functionality #1613

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f8e5d42
Add project clone retries functionality
dkwon17 Apr 9, 2026
d312f58
Update based on PR feedback
dkwon17 Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 26 additions & 4 deletions project-clone/internal/git/setup.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//
// Copyright (c) 2019-2025 Red Hat, Inc.
// Copyright (c) 2019-2026 Red Hat, Inc.
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
Expand All @@ -20,6 +20,7 @@ import (
"log"
"os"
"path"
"time"

dw "github.com/devfile/api/v2/pkg/apis/workspaces/v1alpha2"
projectslib "github.com/devfile/devworkspace-operator/pkg/library/projects"
Expand All @@ -46,9 +47,24 @@ func doInitialGitClone(project *dw.Project) error {
// Clone into a temp dir and then move set up project to PROJECTS_ROOT to try and make clone atomic in case
// project-clone container is terminated
tmpClonePath := path.Join(internal.CloneTmpDir, projectslib.GetClonePath(project))
err := CloneProject(project, tmpClonePath)
if err != nil {
return fmt.Errorf("failed to clone project: %s", err)
var cloneErr error
for attempt := 0; attempt <= internal.CloneRetries; attempt++ {
if attempt > 0 {
delayBeforeRetry(project.Name, attempt)
if err := os.RemoveAll(tmpClonePath); err != nil {
log.Printf("Warning: cleanup before retry failed: %s", err)
}
}
cloneErr = CloneProject(project, tmpClonePath)
if cloneErr == nil {
break
}
if attempt < internal.CloneRetries {
log.Printf("Failed git clone for project %s (attempt %d/%d): %s", project.Name, attempt+1, internal.CloneRetries+1, cloneErr)
}
}
if cloneErr != nil {
return fmt.Errorf("failed to clone project: %w", cloneErr)
}
Comment on lines +66 to 68
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Final clone-failure path skips temp directory cleanup.

After the last failed attempt, Line 67 returns immediately and can leave partial repo data in tmpClonePath. Over repeated init-container failures, stale temp directories may accumulate under the mounted projects volume.

Proposed fix 
 	if cloneErr != nil {
+		if err := os.RemoveAll(tmpClonePath); err != nil {
+			log.Printf("Warning: cleanup after final clone failure failed: %s", err)
+		}
 		return fmt.Errorf("failed to clone project: %w", cloneErr)
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if cloneErr != nil {
return fmt.Errorf("failed to clone project: %w", cloneErr)
}
if cloneErr != nil {
if err := os.RemoveAll(tmpClonePath); err != nil {
log.Printf("Warning: cleanup after final clone failure failed: %s", err)
}
return fmt.Errorf("failed to clone project: %w", cloneErr)
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@project-clone/internal/git/setup.go` around lines 66 - 68, The clone error
return currently exits without removing the temporary clone directory
(tmpClonePath), leaving stale data; ensure tmpClonePath is cleaned on failure by
calling os.RemoveAll(tmpClonePath) before returning (or by registering a defer
cleanup immediately after creating tmpClonePath) in the function that performs
the clone (refer to the tmpClonePath variable and the clone error handling block
that returns fmt.Errorf("failed to clone project: %w", cloneErr)).


if project.Attributes.Exists(internal.ProjectSparseCheckout) {
Expand Down Expand Up @@ -83,6 +99,12 @@ func doInitialGitClone(project *dw.Project) error {
return nil
}

func delayBeforeRetry(projectName string, attempt int) {
delay := internal.BaseRetryDelay * (1 << (attempt - 1))
log.Printf("Retrying git clone for project %s (attempt %d/%d) after %s", projectName, attempt+1, internal.CloneRetries+1, delay)
time.Sleep(delay)
}

func setupRemotesForExistingProject(project *dw.Project) error {
projectPath := path.Join(internal.ProjectsRoot, projectslib.GetClonePath(project))
repo, err := internal.OpenRepo(projectPath)
Expand Down
28 changes: 24 additions & 4 deletions project-clone/internal/global.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//
// Copyright (c) 2019-2025 Red Hat, Inc.
// Copyright (c) 2019-2026 Red Hat, Inc.
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
Expand All @@ -21,6 +21,8 @@ import (
"log"
"os"
"regexp"
"strconv"
"time"

"github.com/devfile/devworkspace-operator/pkg/library/constants"
gittransport "github.com/go-git/go-git/v5/plumbing/transport"
Expand All @@ -30,14 +32,19 @@ import (
)

const (
credentialsMountPath = "/.git-credentials/credentials"
sshConfigMountPath = "/etc/ssh/ssh_config"
publicCertsDir = "/public-certs"
credentialsMountPath = "/.git-credentials/credentials"
sshConfigMountPath = "/etc/ssh/ssh_config"
publicCertsDir = "/public-certs"
cloneRetriesEnvVar = "PROJECT_CLONE_RETRIES"
defaultCloneRetries = 3
maxCloneRetries = 10
BaseRetryDelay = 1 * time.Second
)

var (
ProjectsRoot string
CloneTmpDir string
CloneRetries int
tokenAuthMethod map[string]*githttp.BasicAuth
credentialsRegex = regexp.MustCompile(`https://(.+):(.+)@(.+)`)
)
Expand All @@ -59,6 +66,19 @@ func init() {
log.Printf("Using temporary directory %s", tmpDir)
CloneTmpDir = tmpDir

CloneRetries = defaultCloneRetries
if val := os.Getenv(cloneRetriesEnvVar); val != "" {
parsed, err := strconv.Atoi(val)
if err != nil || parsed < 0 {
log.Printf("Invalid value for %s: %q, using default (%d)", cloneRetriesEnvVar, val, defaultCloneRetries)
} else if parsed > maxCloneRetries {
log.Printf("Value for %s (%d) exceeds maximum (%d), using maximum", cloneRetriesEnvVar, parsed, maxCloneRetries)
CloneRetries = maxCloneRetries
} else {
CloneRetries = parsed
}
}

setupAuth()
}

Expand Down
Loading