Git Remote Integration

How KK handles remote storage with and without Git remotes, and how code files versus large files are managed.

Overview

KK is a version control system designed for projects with large binary assets. It combines:

• Git for version controlling code and pointer files
• Object Remotes (local, rclone, Google Drive) for storing large binary files

When you work with KK:

• Code files are stored directly in git as regular files
• Large files are converted to lightweight pointer files in git
• The actual binary content is stored on object remotes

How KK Works Without Git Remotes

Repository Setup

Without git remotes, KK uses only object remotes for collaboration:

kk init
kk setup gdrive        # or: kk remote add rclone backup --remote gdrive:KK/MyGame

How Files Are Handled

Pushing Without Git Remotes

When you run kk push:

1. Pointer History: Git history is bundled and uploaded to object remotes
2. Large Files: Binary objects are uploaded to object remotes
3. Project Files: Project metadata and config are synced

kk push
# → Uploads binary objects to object remote(s)
# → Bundles and uploads git history to object remote(s)
# → Syncs project files to object remote(s)

Pulling Without Git Remotes

When you run kk pull:

1. History: Downloads and applies history bundles from object remotes
2. Large Files: Downloads binary objects to .kk/objects/
3. Materialization: Expands pointer files to their actual content

kk pull
# → Downloads and applies history bundles
# → Downloads binary objects
# → Materializes pointer files

Cloning Without Git Remotes

kk clone rclone:gdrive:KK/MyGame
# or
kk clone drive:<folder-id>
# or
kk clone local:/path/to/project

Process:

1. Downloads project files from object remote
2. Creates KK directory structure
3. Downloads binary objects (if --pull specified)
4. Initializes .kk/git with bundled history

How KK Works With Git Remotes

Repository Setup

With git remotes, KK uses a hybrid approach:

kk init
kk remote add git origin https://github.com/your-username/MyGame.git
kk setup gdrive        # Object remote for large files

Adding a Git Remote

kk remote add git origin https://github.com/your-username/MyGame.git

KK will:

1. Verify connectivity by running git ls-remote
2. Add the remote to .kk/git/config
3. Store metadata in .kk/config.json

How Files Are Handled

What Gets Pushed to Git vs Object Remotes

Git remote receives:

• All source code files (based on codeExtensions map)
• Pointer files (text references to large files)
• Configuration files (.kk/config.json, .kk/tracks.json)
• Git history

Object remote receives:

• Binary content of large files
• Manifests tracking what objects exist
• (Optional) History bundles if no git remote exists

Pushing With Git Remotes

When you run kk push:

kk push
# → Syncs pointer history to git remote (git push)
# → Uploads binary objects to object remote(s)
# → (Optional) History bundles if no git remote

Pulling With Git Remotes

When you run kk pull:

kk pull
# → Pulls pointer history from git remote (git pull)
# → Downloads binary objects from object remote(s)
# → Materializes pointer files

Cloning From Git

kk clone git:https://github.com/your-username/MyGame.git
kk clone git:https://github.com/your-username/MyGame.git --pull
kk clone git:https://github.com/your-username/MyGame.git --account myprofile --pull
kk clone git:https://gitlab.com/your-username/MyGame.git --branch kk-test --pull

Process:

1. Git clone: Uses standard git to clone the repository
2. File copy: Copies files (excluding .git directory) to destination
3. KK setup: Initializes KK structure if not present
4. Remote config: Adds git remote to .kk/git
5. Object check: Verifies object remotes accessibility
6. Materialization: Downloads and expands pointer files (if --pull)

What you get:

• All source code files as regular files
• Pointer files for large assets
• .kk/config.json with remote configurations
• .kk/tracks.json with tracking patterns

Code Files vs Large Files

Code Files

Code files are defined by the codeExtensions map in internal/core/code.go. These files are always stored as regular files in git:

... and 100+ more extensions.

Large Files (Pointers)

Files that match patterns in .kk/tracks.json and are NOT code files are converted to pointers:

Pointer format:

version kk-lfs-1.0.0
oid sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
size 104857600

Example tracking patterns:

{
  "patterns": [
    "*.mp4",
    "*.wav",
    "*.mp3",
    "*.png",
    "*.jpg",
    "*.jpeg",
    "*.psd",
    "*.fbx",
    "*.obj",
    "*.blend",
    "*.zip",
    "*.tar.gz"
  ]
}

Adding Files

# Add code file - stored as regular file
kk add src/main.go
# Output: code src/main.go (storing as regular file)

# Add large file - converted to pointer
kk add Assets/video.mp4
# Output: large Assets/video.mp4 -> sha256:9f86... (104857600 bytes)

Why This Matters

• GitHub/GitLab/Gitea can see your source code normally
• Repository size stays small - only text files in git
• Large files are accessible from object remotes
• Collaborators can clone with regular git and get code, then use KK for assets

Repository Structure

MyProject/
├── .kk/                          # KK metadata
│   ├── git/                      # KK's internal git repository
│   │   ├── HEAD
│   │   ├── objects/              # Git objects
│   │   ├── refs/                 # Git references
│   │   └── config                # Git config (includes git remotes)
│   ├── objects/                  # Local cache of large files
│   │   └── ab/cd/<hash>         # Content-addressed storage
│   ├── config.json               # Remote configurations
│   ├── tracks.json               # File tracking patterns
│   ├── repo.json                 # Repository metadata (repo_id, name)
│   ├── tmp/                      # Temporary files
│   └── logs/                     # Operation logs
├── .kkignore                     # Exclusion patterns
├── src/                          # Source code (stored in git)
│   └── main.go
├── Assets/                       # Large assets (pointers in git)
│   ├── video.mp4                 # Pointer file
│   └── model.fbx                 # Pointer file
└── README.md                     # Documentation (stored in git)

How To: Set Up Git Remote + Google Drive

This step-by-step guide shows you how to set up KK with both a Git hosting service (GitHub, GitLab, or Gitea) for code/history and Google Drive for large files. This is the most common setup for teams.

Prerequisites

Before you start, make sure you have:

• KK installed — See Installation Guide
• A Git hosting account — GitHub, GitLab, or a self-hosted Gitea instance
• A Google account — For Google Drive storage
• Git installed — KK uses Git under the hood for version control

Step 1: Create your Git repository

First, create an empty repository on your Git hosting service:

For GitHub:

1. Go to github.com/new
2. Name your repository (e.g., my-game)
3. Important: Don't initialize with README, .gitignore, or license — KK will handle this
4. Click "Create repository"
5. Copy the repository URL (e.g., https://github.com/username/my-game.git)

For GitLab:

1. Go to your project's "New project" page
2. Create a blank project
3. Copy the repository URL

For Gitea (self-hosted):

1. Create a new repository on your Gitea instance
2. Copy the repository URL

Step 2: Initialize your KK project

# Navigate to your project directory
cd my-game

# Initialize KK
kk init

This creates the .kk/ directory structure with embedded Git database.

Step 3: Add the Git remote

Add the Git repository you created in Step 1:

# For GitHub
kk remote add git origin https://github.com/your-username/my-game.git

# For GitLab
kk remote add git origin https://gitlab.com/your-username/my-game.git

# For Gitea (self-hosted)
kk remote add git origin https://git.example.com/your-username/my-game.git

What this does:

• Verifies the Git remote is accessible
• Adds the remote to KK's internal Git configuration
• Stores the remote metadata in .kk/config.json

kk remote add git origin git@github.com:your-username/my-game.git

Step 4: Set up Google Drive

kk setup gdrive

This will:

1. Open your browser for Google OAuth authorization
2. Ask you to sign in with your Google account
3. Request permission to access Google Drive
4. Create a project folder in your Google Drive

After successful authorization, you'll see:

kk: Google Drive auth saved to ~/.config/KK/gdrive/default.json
kk: Drive remote 'gdrive' configured with folder-id: 1a2B3cD4eF5gH6iJ7kL8mN9oP0qR1sT2u
kk: Run 'kk push' to upload your project to Drive.
kk: Remote ready.

Step 5: Track large file types (Optional)

Note: This step is optional! By default, if no custom tracking patterns are configured, KK automatically tracks all files that are not recognized as code (such as .unitypackage, .uasset, .fbx, .png, .wav, etc.) as pointers.

Explicitly running kk track is only required if you want to define custom, narrow tracking patterns (which will turn off the default "track all non-code" behavior). If you wish to specify custom patterns, use the commands below:

# Common game development assets
kk track "*.mp4" "*.wav" "*.mp3"    # Audio/video
kk track "*.png" "*.jpg" "*.psd"    # Images/textures
kk track "*.fbx" "*.obj" "*.blend"  # 3D models
kk track "*.zip" "*.tar.gz"         # Archives

What gets tracked: Files matching these patterns will be converted to pointer files in Git, with the actual content stored on Google Drive.

What stays as regular files: Source code (.go, .py, .js, etc.), config files (.json, .yaml), and text files are stored directly in Git as regular files.

Step 6: Add and commit your files

# Add all files in your project
kk add .

# Commit your changes
kk commit -m "Initial commit"

What happens during commit:

• Code files are stored directly in Git
• Large files are converted to pointers and stored in .kk/objects/
• The pointer references are committed to Git

Step 7: Push to both remotes

kk push

This single command does both of the following simultaneously:

• Pushes to Git: Sends code, pointers, and history to GitHub/GitLab/Gitea via git push
• Pushes to Drive: Uploads large file binaries to your Google Drive folder

kk remote remove gdrive
kk remote add drive gdrive --drive-folder-id <id> --drive-auth-path <path> --verify-mode none

Step 8: Share with teammates

A. Share the Git repository

Invite collaborators to your Git repository through your Git hosting service's interface (GitHub Settings → Collaborators, GitLab Members, etc.).

B. Share the Google Drive folder

1. Open Google Drive and find the KK project folder
2. Right-click the folder → "Share"
3. Add teammates' Google account emails with "Editor" permissions
4. Share the folder ID with your team

C. Teammates clone the project

# Teammate runs this command
kk clone git:https://github.com/your-username/my-game.git --pull

This will:

1. Clone the Git repository (code + pointers)
2. Download large files from Google Drive (if shared)
3. Set up both remotes automatically

Complete Working Example

Here's the complete sequence from start to finish:

# 1. Navigate to project
cd ~/Projects/my-game

# 2. Initialize KK
kk init

# 3. Add Git remote (GitHub example)
kk remote add git origin https://github.com/johndoe/my-game.git

# 4. Set up Google Drive (opens browser for OAuth)
kk setup gdrive

# 5. Track large file types
kk track "*.mp4" "*.wav" "*.png" "*.fbx" "*.blend"

# 6. Add files and commit
kk add .
kk commit -m "Initial commit with game assets"

# 7. Push to both remotes
kk push

# 8. Share with team
# - Share GitHub repo via GitHub's interface
# - Share Drive folder via Google Drive's interface
# - Tell teammates: kk clone git:https://github.com/johndoe/my-game.git --pull

Daily Workflow

After initial setup, your daily workflow is simple:

# Make changes to your files...

# Add new or changed files
kk add .

# Commit changes
kk commit -m "Add new character model"

# Push to both Git and Drive
kk push

Troubleshooting Common Issues

Alternative: Use rclone for Cloud Storage

If you prefer to use rclone instead of native Google Drive, or want to use other cloud providers like Dropbox, MEGA, S3, OneDrive, or any rclone-supported service, follow this guide.

Step 1: Install rclone

Important: KK looks for rclone (or rclone.exe on Windows) in your system PATH. You can also specify a custom path when adding the remote.

Windows (Recommended for regular users)

1. Download rclone from rclone.org/downloads
2. Find the Windows amd64 download link (looks like rclone-x.x.x-windows-amd64.zip)
3. Download and unzip the file
4. Copy rclone.exe to one of these locations:

rclone version

Quick install with package managers:

# Windows with Chocolatey (if installed)
choco install rclone

# Windows with Scoop (if installed)
scoop install rclone

# macOS
brew install rclone

# Linux
curl https://rclone.org/install.sh | sudo bash

Step 2: Configure your cloud provider in rclone

Run the rclone configuration wizard. This is a one-time setup per provider:

rclone config

You'll see an interactive prompt. Follow these steps:

1. Type n for "New remote"
2. Enter a name (e.g., gdrive, dropbox, mega)
3. Choose the storage type from the list (number)
4. Follow provider-specific instructions (often opens a browser for OAuth)
5. Choose n for "No advanced config" (keep it simple!)
6. Type y to confirm, then q to quit

Common provider numbers in rclone:

Step 3: Test rclone connection

Before adding to KK, verify rclone can access your cloud storage:

# List root directories (tests connection)
rclone lsd gdrive:

# Create a test folder
rclone mkdir gdrive:KK-Test

If you see your folders listed, rclone is configured correctly!

Step 4: Add rclone remote to KK

With rclone configured and tested, add it to KK:

# Basic form (simplest)
kk remote add rclone mycloud --remote gdrive:KK/MyGame --push true --pull true

What the parameters mean:

• mycloud — A name you choose to reference this remote
• --remote — The rclone path (provider-name:folder-path)
• --push true — Allow uploading files to this remote
• --pull true — Allow downloading files from this remote

Full form with all options:

kk remote add rclone gdrive \
  --display-name "Google Drive" \
  --role backup \
  --provider google-drive \
  --remote gdrive:KK/MyGame \
  --verify-mode download \
  --priority 20 \
  --push true \
  --pull true

Custom rclone binary location (Windows):

If rclone.exe is not on your PATH, specify the full path to the binary when adding the remote:

# Windows example with full path
kk remote add rclone mycloud \
  --remote "mega:KK/MyGame" \
  --binary "C:\Tools\rclone\rclone.exe" \
  --push true \
  --pull true

--binary rclone.exe

Complete Example: GitHub + Dropbox (via rclone)

Here's a complete workflow using GitHub for code and Dropbox for files:

# 1. Initialize KK
cd ~/Projects/my-game
kk init

# 2. Add GitHub remote for code
kk remote add git origin https://github.com/your-username/my-game.git

# 3. Configure Dropbox in rclone (one-time setup)
rclone config
# → Choose "dropbox" storage type
# → Follow OAuth in browser
# → Name it "dropbox"

# 4. Test Dropbox connection
rclone lsd dropbox:

# 5. Add Dropbox to KK
kk remote add rclone dropbox --remote dropbox:KK-Projects/MyGame --push true --pull true

# 6. Track large files
kk track "*.mp4" "*.wav" "*.fbx" "*.blend"

# 7. Add and commit
kk add .
kk commit -m "Initial commit"

# 8. Push to both GitHub and Dropbox
kk push

Complete Example: GitHub + MEGA (via rclone)

# 1. Initialize
kk init

# 2. Add GitHub
kk remote add git origin https://github.com/your-username/my-game.git

# 3. Configure MEGA in rclone
rclone config
# → Choose "mega" storage type
# → Enter your MEGA credentials
# → Name it "mega"

# 4. Test MEGA
rclone lsd mega:

# 5. Add MEGA to KK
kk remote add rclone mega --remote mega:KK/MyGame --push true --pull true

# 6. Set up and push
kk track "*.psd" "*.fbx" "*.mp4"
kk add .
kk commit -m "Add game assets"
kk push

Complete Example: GitHub + S3 (via rclone)

# 1. Initialize
kk init

# 2. Add GitHub
kk remote add git origin https://github.com/your-username/my-game.git

# 3. Configure S3 in rclone
rclone config
# → Choose "s3" storage type
# → Choose your S3 provider (AWS, Wasabi, Backblaze, etc.)
# → Enter access key and secret
# → Name it "s3"

# 4. Test S3
rclone lsd s3:

# 5. Add S3 to KK
kk remote add rclone s3backup --remote s3:my-kk-bucket/MyGame --push true --pull true

# 6. Set up and push
kk track "*.zip" "*.tar.gz" "*.mp4"
kk add .
kk commit -m "Initial commit"
kk push

Multiple remotes with rclone (redundancy)

You can use multiple rclone remotes for backup and redundancy:

# Add primary storage (Dropbox)
kk remote add rclone dropbox-primary \
  --remote dropbox:KK/MyGame \
  --priority 10 \
  --push true \
  --pull true

# Add backup storage (MEGA)
kk remote add rclone mega-backup \
  --remote mega:KK/MyGame \
  --priority 50 \
  --push true \
  --pull true

# Set default remote
kk remote set-default dropbox-primary

# Push to all remotes
kk push --all-remotes

Troubleshooting rclone issues

Next Steps

• How It Works — Learn more about KK's internals
• How To Guide — More guides for advanced workflows
• Glossary — Understand KK terminology

Common Workflows

Scenario 1: Solo Developer, Local Storage

# Initialize
kk init
kk track "*.mp4" "*.fbx" "*.blend"

# Work
kk add src/main.go Assets/video.mp4
kk commit -m "Add video player"

# Backup to NAS
kk remote add local nas --path /Volumes/NAS/KK/MyProject
kk push --remote nas

Scenario 2: Team with GitHub + Google Drive

# Initialize (developer 1)
kk init
kk remote add git origin https://github.com/team/project.git
kk setup gdrive
kk track "*.mp4" "*.fbx" "*.blend"

# Work
kk add src/main.go Assets/video.mp4
kk commit -m "Initial commit"
kk push  # Pushes code to GitHub, binaries to Drive

# Clone (developer 2)
kk clone git:https://github.com/team/project.git --pull
# → Gets code from GitHub
# → Downloads binaries from Drive

Scenario 3: Team Without GitHub

# Initialize
kk init
kk setup gdrive

# Work
kk add src/main.go Assets/video.mp4
kk commit -m "Initial commit"
kk push  # Pushes everything to Drive (includes history bundles)

# Clone
kk clone drive:<folder-id> --pull
# → Gets everything from Drive

Scenario 4: Using Existing Git Repository

# Clone existing repo
git clone https://github.com/team/project.git
cd project

# Initialize KK
kk init --here

# Setup remotes
kk remote add git origin https://github.com/team/project.git
kk setup gdrive

# Track large files
kk track "*.mp4" "*.fbx"

# Add files
kk add .  # Code files stay regular, large files become pointers
kk commit -m "Add large assets with KK"
kk push

Migrating Between History Modes

KK stores commit history in exactly one of two modes. You can switch at any time using kk remote migrate.

Scenario 5: Storage bundles → GitHub (to-git)

A solo developer has been using Google Drive bundles. The team grows and wants GitHub for code review.

# Step 1 — verify current mode (no git remote listed)
kk remote list
# default: nas
# nas type=local provider=nas pull=true push=true priority=10

# Step 2 — migrate (connectivity is checked before any config changes)
kk remote migrate to-git github https://github.com/acme/my-game.git

Output:

kk: checking git remote accessibility...
kk: pushing all local branches to "github"...
To https://github.com/acme/my-game.git
 * [new branch]  main -> main
 * [new branch]  feature/ai -> feature/ai
kk: [github] all branches pushed

kk: migration complete → git remote "github" registered (https://github.com/acme/my-game.git)
kk: future 'kk push' will sync pointer history to "github" via git push
kk: existing history bundles on object remote(s) are kept but will not be extended

# Step 3 — confirm new state
kk remote list
# default: nas
# nas    type=local provider=nas    pull=true push=true priority=10
# github type=git  provider=github  url=https://github.com/acme/my-game.git push=true pull=true

# Step 4 — from now on, kk push syncs via git push
kk push
# kk: [github] syncing pointer history to git remote...
# kk: [github] pointer history synced
# kk: [nas] ...

# Step 5 — teammates clone from GitHub
kk clone git:https://github.com/acme/my-game.git --pull

If auth isn't set up yet

kk remote migrate to-git registers the remote even if the initial push fails (e.g. SSH key not yet added to GitHub). The recovery command is printed:

kk: warning: initial git push failed: exit status 128
kk:          Once auth is set up, run:
kk:          git --git-dir=.kk/git push --all github

Scenario 6: GitHub → Storage bundles (to-storage)

The team is leaving GitHub and going to a self-hosted NAS only.

# Step 1 — verify current mode
kk remote list
# default: github
# github type=git  provider=github  url=https://github.com/acme/my-game.git push=true pull=true
# nas    type=local provider=nas    pull=true push=true priority=10

# Step 2 — preview the migration (shows what will change, asks to confirm)
kk remote migrate to-storage

Output:

kk: This will:
kk:   1. Upload a full history bundle to: nas
kk:   2. Remove git remote(s): github (https://github.com/acme/my-game.git)
kk:   Future 'kk push' will upload incremental bundles instead of using git push.

Proceed? [y/N] y
kk: creating initial history bundle(s)...
kk: [nas] creating history bundle (full.bundle)...
kk: [nas] uploading history bundle...
kk: [nas] history pushed (full.bundle, 2 branch(es))
kk: default remote set to "nas"

kk: migration complete → commit history will now travel via storage bundles
kk: teammates can restore history with: kk clone <spec> --history

# Step 3 — confirm new state
kk remote list
# default: nas
# nas type=local provider=nas pull=true push=true priority=10

# Step 4 — future pushes create incremental bundles automatically
kk push
# kk: [nas] creating history bundle (inc-000001.bundle)...
# kk: [nas] history pushed (inc-000001.bundle, 2 branch(es))

# Step 5 — teammates clone from NAS with full history
kk clone local:/NAS/KK/my-game --history --pull

Targeting one git remote when multiple exist

# Remove only the gitlab remote; keep github
kk remote migrate to-storage --remote gitlab --yes

# After this, github still exists and handles history;
# gitlab is removed from both .kk/git and .kk/config.json

Scripted / non-interactive use

# Skip the confirmation prompt (useful in CI or automation)
kk remote migrate to-storage --yes

Idempotency & Safety Guarantees

Both commands are safe to run more than once. If the project is already in the target mode, the command prints a notice and exits cleanly — no error, no changes.

Configuration

.kk/config.json

{
  "version": "kk-local-config-1.0.0",
  "default_remote": "origin",
  "remotes": {
    "origin": {
      "type": "git",
      "url": "https://github.com/your-username/MyGame.git",
      "provider": "github",
      "pull": true,
      "push": true,
      "tags": ["git"]
    },
    "gdrive-backup": {
      "type": "drive",
      "drive_folder_id": "1ABC...",
      "drive_auth_path": "...",
      "provider": "google-drive",
      "pull": true,
      "push": true,
      "priority": 10
    },
    "nas": {
      "type": "local",
      "path": "/Volumes/NAS/KK/MyProject",
      "verify_mode": "local-hash",
      "pull": true,
      "push": true,
      "priority": 20
    }
  }
}

Remote Types

Remote Priority

Lower numbers = higher priority. KK tries remotes in order when downloading objects:

priority: 10  → Studio NAS (fast, local)
priority: 20  → Google Drive (slower, cloud)
priority: 50  → Archive storage (slowest)

See Also

• How It Works — end-to-end workflow and internals
• Glossary — term definitions
• How To Guide — step-by-step user guides