🛠 Developer Handoff

GUI → CLI Companion

A macOS menu bar app that shows the terminal equivalent of what you're doing in the GUI — in real time. Technical spec for the Swift implementation.

Target OS
macOS 14+
Language
Swift 5.9
Frameworks
SwiftUI + AppKit
Updated
2026-05-02
📄 This file lives at docs/handoff.html — edit it directly in your repo to keep the spec up to date. Code snippets can be copy-pasted straight into Xcode.

00 / Live Commands

Ansluter till gui_to_cli.py på port 7070…
Inga händelser ännu — kör python3 tools/mqmirror/gui_to_cli.py och gör saker i macOS GUI.

01 / Architecture

Data flow
macOS GUI
Event Sources
EventBus
CommandMapper
SwiftUI UI
Event sources
NSWorkspace
app switches, launches, disk mounts — no permission needed
AXObserver
button presses, menu selections, focus changes — requires Accessibility
FSEvents
file create / delete / rename — requires Full Disk Access
CGEventTap
global keyboard shortcuts (Cmd+C, Cmd+N…) — requires Input Monitoring
📦

App type

LSUIElement = true — no Dock icon. Lives as NSStatusItem in the menu bar.

🔄

State

@Observable CommandStore holds history. SwiftUI views subscribe via @Environment.

🗺

CommandMapper

Static dictionary + plugin architecture. Each app (Finder, Safari…) is its own Mapper struct.

02 / macOS APIs

Core

NSWorkspace Notifications

Catches app launch, activation, disk mount, sleep/wake. No extra permission needed.

didActivateApplicationNotification didLaunchApplicationNotification didMountNotification
Core

AXObserver (Accessibility API)

Listens to UI elements in all apps: button presses, menu selections, focus changes. Requires Accessibility permission.

AXUIElementCreateApplication AXObserverCreate kAXPressedNotification kAXFocusedUIElementChangedNotification
Core

FSEvents

Real-time filesystem events for chosen directories. Catches create, delete, move, rename.

FSEventStreamCreate kFSEventStreamEventFlagItemCreated kFSEventStreamEventFlagItemRenamed kFSEventStreamEventFlagItemRemoved
Optional

CGEventTap

Global keyboard and mouse events. Catches Cmd+C, Cmd+N, Cmd+Z and other shortcuts. Requires Input Monitoring.

CGEvent.tapCreate kCGEventKeyDown CGEventFlags
Optional — v2

NSAppleScript / JXA

Query Finder directly: selected files, current directory, Trash contents. Gives rich context for the command mapper.

NSAppleScript.executeAndReturnError com.apple.security.automation.apple-events
Optional — v2

NSStatusItem (Menu Bar)

App lives in the menu bar. SwiftUI popover shown on click. No Dock icon.

NSStatusBar.system.statusItem NSPopover LSUIElement = true

03 / Permissions & Entitlements

Entitlement / Permission Why When to request Required
Accessibility (AX) Read UI elements in other apps via AXObserver First launch Required
Full Disk Access FSEvents in protected directories (Desktop, Downloads) First launch Required
Input Monitoring CGEventTap for global keyboard shortcuts On feature enable Optional
com.apple.security.automation.apple-events NSAppleScript to query Finder for context On feature enable Optional
App Sandbox Disable for full system access (distribute outside App Store) Build config Build setting

Onboarding strategy

Request permissions progressively — explain each one right before asking for it.

  • Launch → explain the app, show onboarding view
  • Step 1: Accessibility → open System Settings automatically with AXIsProcessTrustedWithOptions
  • Step 2: Full Disk Access → deep-link to the correct System Settings pane
  • Step 3 (opt): Input Monitoring → expose as toggle in Settings view
  • Step 4 (opt): Apple Events → expose as toggle in Settings view

04 / Code

Swift AppDelegate.swift
@main
struct GUItoCLIApp: App {
    @NSApplicationDelegateAdaptor(AppDelegate.self) var delegate
    var body: some Scene {
        // No main window — app lives in the menu bar
        Settings { SettingsView() }
    }
}

class AppDelegate: NSObject, NSApplicationDelegate {
    var statusItem: NSStatusItem!
    var popover = NSPopover()
    let store = CommandStore()

    func applicationDidFinishLaunching(_ notification: Notification) {
        statusItem = NSStatusBar.system.statusItem(
            withLength: NSStatusItem.squareLength)
        statusItem.button?.image = NSImage(
            systemSymbolName: "terminal",
            accessibilityDescription: "GUI→CLI")
        statusItem.button?.action = #selector(togglePopover)

        popover.contentSize = NSSize(width: 420, height: 560)
        popover.behavior = .transient
        popover.contentViewController = NSHostingController(
            rootView: CompanionView().environmentObject(store))

        WorkspaceMonitor(store: store).start()
        AccessibilityMonitor(store: store).start()
        FileSystemMonitor(store: store).start()
    }

    @objc func togglePopover() {
        if popover.isShown { popover.close() }
        else if let btn = statusItem.button {
            popover.show(relativeTo: btn.bounds, of: btn,
                preferredEdge: .minY)
        }
    }
}
Swift CommandStore.swift
struct CLICommand: Identifiable {
    let id        = UUID()
    let timestamp:  Date
    let guiAction:  String    // "Created folder on Desktop"
    let command:    String    // "mkdir ~/Desktop/folder"
    let category:   String    // "Finder" | "System" | …
    let flags:      [Flag]    // token + explanation pairs
    let related:    [String]  // related commands
}

struct Flag {
    let token: String
    let kind:  FlagKind  // .cmd | .flag | .path | .arg
    let desc:  String
}

@Observable
class CommandStore {
    var history:  [CLICommand] = []
    var current:  CLICommand?
    var isFrozen: Bool = false

    func push(_ cmd: CLICommand) {
        guard !isFrozen else { return }
        Task { @MainActor in
            current = cmd
            history.insert(cmd, at: 0)
            if history.count > 100 { history.removeLast() }
        }
    }
}
Swift WorkspaceMonitor.swift
class WorkspaceMonitor {
    let store: CommandStore
    let mapper = AppLaunchMapper()

    func start() {
        let nc = NSWorkspace.shared.notificationCenter
        nc.addObserver(self,
            selector: #selector(appActivated(_:)),
            name: NSWorkspace.didActivateApplicationNotification,
            object: nil)
        nc.addObserver(self,
            selector: #selector(diskMounted(_:)),
            name: NSWorkspace.didMountNotification,
            object: nil)
    }

    @objc func appActivated(_ n: Notification) {
        guard let app = n.userInfo?[NSWorkspace.applicationUserInfoKey]
            as? NSRunningApplication,
            let name = app.localizedName,
            let cmd = mapper.command(forApp: name)
        else { return }
        store.push(cmd)
    }

    @objc func diskMounted(_ n: Notification) {
        guard let name = n.userInfo?["NSWorkspaceVolumeLocalizedNameKey"]
            as? String else { return }
        store.push(CLICommand(
            timestamp: .now(),
            guiAction: "Mount disk: \(name)",
            command:   "diskutil mount /Volumes/\(name)",
            category:  "System", flags: [],
            related:   ["hdiutil attach disk.dmg"]
        ))
    }
}
Swift FileSystemMonitor.swift
class FileSystemMonitor {
    let store: CommandStore
    var stream: FSEventStreamRef?

    let watchedPaths = ["~/Desktop", "~/Documents", "~/Downloads"]
        .map { ($0 as NSString).expandingTildeInPath }

    func start() {
        var ctx = FSEventStreamContext(
            version: 0,
            info: Unmanaged.passUnretained(self).toOpaque(),
            retain: nil, release: nil, copyDescription: nil)

        stream = FSEventStreamCreate(
            nil,
            { _, info, numEvents, eventPaths, eventFlags, _ in
                let monitor = Unmanaged<FileSystemMonitor>
                    .fromOpaque(info!).takeUnretainedValue()
                let paths = unsafeBitCast(eventPaths, to: [String].self)
                for i in 0..<numEvents {
                    monitor.handleEvent(path: paths[i], flags: eventFlags[i])
                }
            },
            &ctx,
            watchedPaths as CFArray,
            FSEventStreamEventId(kFSEventStreamEventIdSinceNow),
            0.5,
            FSEventStreamCreateFlags(
                kFSEventStreamCreateFlagFileEvents |
                kFSEventStreamCreateFlagUseCFTypes))

        if let s = stream {
            FSEventStreamScheduleWithRunLoop(
                s, CFRunLoopGetMain(), CFRunLoopMode.defaultMode.rawValue)
            FSEventStreamStart(s)
        }
    }

    func handleEvent(path: String, flags: FSEventStreamEventFlags) {
        let name = (path as NSString).lastPathComponent
        guard !name.hasPrefix(".") else { return }

        let isDir = flags & FSEventStreamEventFlags(
            kFSEventStreamEventFlagItemIsDir) != 0

        if flags & FSEventStreamEventFlags(
            kFSEventStreamEventFlagItemCreated) != 0 {
            store.push(CLICommand(
                timestamp: .now(),
                guiAction: "New \(isDir ? "folder" : "file"): \(name)",
                command: isDir
                    ? "mkdir \"\(path)\""
                    : "touch \"\(path)\"",
                category: "Finder", flags: [], related: []))
        } else if flags & FSEventStreamEventFlags(
            kFSEventStreamEventFlagItemRemoved) != 0 {
            store.push(CLICommand(
                timestamp: .now(),
                guiAction: "Deleted: \(name)",
                command: "rm -r \"\(path)\"",
                category: "Finder", flags: [],
                related: ["trash \"\(path)\""]))
        } else if flags & FSEventStreamEventFlags(
            kFSEventStreamEventFlagItemRenamed) != 0 {
            store.push(CLICommand(
                timestamp: .now(),
                guiAction: "Renamed/moved: \(name)",
                command: "mv \"old-path\" \"\(path)\"",
                category: "Finder", flags: [], related: []))
        }
    }
}
Swift AccessibilityMonitor.swift
import ApplicationServices

// Maps button/menu titles → CLI commands
let axButtonMap: [String: CLICommand] = [
    "Empty Trash": CLICommand(timestamp: .now(),
        guiAction: "Empty Trash",
        command: "rm -rf ~/.Trash/*",
        category: "Finder", flags: [],
        related: ["osascript -e 'tell application \"Finder\" to empty trash'"]),
    "New Folder": CLICommand(timestamp: .now(),
        guiAction: "New Folder",
        command: "mkdir new-folder",
        category: "Finder", flags: [], related: ["mkdir -p a/b/c"]),
    // … add more mappings here
]

class AccessibilityMonitor {
    let store: CommandStore
    var observer: AXObserver?

    func start() {
        guard AXIsProcessTrusted() else { return }
        let pid = NSWorkspace.shared.frontmostApplication!.processIdentifier
        AXObserverCreate(pid, { _, element, _, _ in
            // Note: C callback — use a global/static store reference
            var title: CFTypeRef?
            AXUIElementCopyAttributeValue(element,
                kAXTitleAttribute as CFString, &title)
            if let t = title as? String, let cmd = axButtonMap[t] {
                CommandStore.shared.push(cmd)
            }
        }, &observer)
    }
}

05 / UI Specification

Popover (primary view)

420 × 560 pt. Three zones:

  • Header — icon + app name + category badge + live indicator
  • Command card — syntax-coloured command, copy button, flag chips (tap to expand)
  • History list — last 20 events, tappable to expand

Settings (native SwiftUI Settings scene)

  • Watched directories (add / remove)
  • Colour theme (Noir / Amber / Phosphor)
  • Permission status with one-click fix buttons
  • Enable / disable individual event sources
  • Launch at login

Typography & Colours

UI Text
.SF Pro / -apple-system
Titles 15/600 · Body 13/400 · Labels 11/500
Monospace
.SF Mono / Menlo
Commands 13px · Related 11px
Syntax colours
.cmd — #79c0ff
.flag — #ff7b72
.path — #a5d6ff
.arg — #7ee787

06 / Roadmap

MVP v1.0
Proof of concept
  • NSWorkspace: app activation → CLI mapping
  • FSEvents: Desktop + Documents + Downloads
  • Menu bar popover with SwiftUI
  • Copy button + history (20 entries)
  • Onboarding for Accessibility + Full Disk Access
Rich events v1.5
AX + Finder context
  • AXObserver: button presses, menu selections, focus changes
  • NSAppleScript: fetch selected files from Finder
  • Dynamic commands with actual file paths
  • CGEventTap: global keyboard shortcuts
  • Flag explanations in popover (tap-to-expand chips)
Smart v2.0
AI + plugins
  • LLM backend (local Ollama) for unknown actions
  • Plugin API: third-party developers can add mappers
  • Export: save session as a shell script
  • Search in history
  • iCloud sync of command history

07 / Changelog

Update this section whenever the spec changes. Keep entries in reverse-chronological order.

2026-05-02
Added Initial spec — architecture, all APIs, Swift code scaffolds, roadmap