Expose a Local Service with Cloudflare Tunnel

Expose a Local Service with Cloudflare Tunnel

Expose a service running on your local machine to a remote server without opening any ports. For instance, let your OpenClaw agent (the remote server) access qBittorrent Web UI on your Mac (the local machine), to download a movie for you.

The local machine makes an outbound-only connection to Cloudflare. The remote server hits your subdomain on Cloudflare's edge. Traffic flows:

OpenClaw on your remote server -> https://your-tunnel-name.example.com -> Cloudflare edge servers -> Cloudflare Tunnel -> qBittorrent Web UI on your local machine

You can probably do the same thing with Tailscale, but unfortunately, Tailscale app doesn't work well with Mullvad VPN on macOS (and I don't want to use Tailscale's Mullvad VPN add-on).

ref:
https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/
https://tailscale.com/docs/features/exit-nodes/mullvad-exit-nodes

Setup

1. Create Cloudflare Tunnel

Do this from any device where you're logged into Cloudflare. No login needed on the local machine or the remote server.

  1. Go to Cloudflare Zero Trust dashboard
  2. Networks -> Connectors -> Create a tunnel -> Cloudflared
    • Name your tunnel: your-tunnel-name
  3. Copy the tunnel token
  4. Configure the tunnel you just created -> Published application routes -> Add a published application route
    • Subdomain: your-tunnel-name
    • Domain: select your domain from the dropdown (e.g., example.com)
    • Path: [leave empty]
    • Service:
      • Type: HTTP
      • URL: localhost:8080
  5. After you create the published application route, Cloudflare will automatically create the DNS record for your subdomain

ref:
https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/get-started/tunnel-useful-terms/
https://developers.cloudflare.com/cloudflare-one/networks/routes/add-routes/

2. Access Controls for Cloudflare Tunnel

Still in the Cloudflare Zero Trust dashboard.

  1. Access controls -> Service credentials -> Service Tokens -> Create Service Token
    • Token name: your-token-name
    • Service Token Duration: Non-expiring
    • Save the CF-Access-Client-Id and CF-Access-Client-Secret (shown only once)
  2. Access controls -> Policies -> Add a policy
    • Policy name: your-policy-name
    • Action: Service Auth
    • Session duration: 24 hours
    • Configure rules -> Include:
      • Selector: Service Token
      • Value: select the service token you just created (e.g., your-token-name)
  3. Access controls -> Applications -> Add an application -> Self-hosted
    • Application name: your-tunnel-name
    • Session Duration: 24 hours
    • Add public hostname:
      • Input method: Default
      • Subdomain: your-tunnel-name (must match the subdomain in step 1.4)
      • Domain: select your domain from the dropdown (e.g., example.com)
      • Path: [leave empty]
    • Select existing policies (this text is a clickable button, not a label!)
      • Check the policy you created in step 2.2

ref:
https://developers.cloudflare.com/cloudflare-one/access-controls/service-credentials/service-tokens/
https://developers.cloudflare.com/cloudflare-one/access-controls/policies/
https://developers.cloudflare.com/cloudflare-one/access-controls/applications/http-apps/

3. Run cloudflared on Local Machine (macOS)

Make cloudflared run on boot, connecting outbound to Cloudflare. No browser auth ever needed.

brew install cloudflared

# install as a LaunchAgent using the tunnel token from step 1
sudo cloudflared service install YOUR_TUNNEL_TOKEN

ref:
https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/downloads/

To verify it's running:

sudo launchctl list | grep cloudflared

4. Access the Local Service on Remote Server

Test that the tunnel and access policy work. We're accessing qBittorrent Web UI here:

curl \
  -H "CF-Access-Client-Id: $YOUR_CF_ACCESS_CLIENT_ID" \
  -H "CF-Access-Client-Secret: $YOUR_CF_ACCESS_CLIENT_SECRET" \
  -d "username=YOUR_USERNAME&password=YOUR_PASSWORD" \
  https://your-tunnel-name.example.com/api/v2/auth/login

The CF-Access-XXX headers must be included on every request. Without them, Cloudflare returns a 302 redirect to a login page.

ref:
https://github.com/qbittorrent/qBittorrent/wiki/#webui

Why Cloudflare Tunnel Over Tailscale

  • No login on endpoints: The tunnel token is scoped to one tunnel, can't access your Cloudflare account
  • No VPN conflicts: cloudflared is just outbound HTTPS, Mullvad VPN doesn't care
  • Free: Cloudflare Zero Trust free tier covers this
Claude Code: Things I Learned After Using It Every Day

Claude Code: Things I Learned After Using It Every Day

I've used Claude Code daily since it came out. Here are the best practices, tools, and configuration patterns I've picked up. Most of this applies to Codex too.

TL;DR
My opinionated setup for Claude Code:
https://github.com/vinta/hal-9000

CLAUDE.md

The Global CLAUDE.md

Your ~/.claude/CLAUDE.md should only contain:

  • Your preferences and nudges to correct agent behaviors
  • You probably don't need to tell it YAGNI or KISS as bare principles. They're already built in.

Pro tip 1: before adding something to CLAUDE.md, ask it, "Is this already covered in your system prompt?"
Pro tip 2: try my refactor-claude-md skill!

Here are some parts of my CLAUDE.md I found useful:

## Communication Style

### Surface Assumptions

Proceeding on a guess is right; proceeding silently is not.

Where my request left a decision open and you resolved it by guessing, name that assumption as its own bullet so I can catch what I forgot to tell you.

When I ask for advice or a recommendation, first surface the assumptions my question takes for granted and the missing information that would change your answer (and how), so I can catch the framing I got wrong.

### Use AskUserQuestion

When you ask the user anything whose answer is a selection rather than a sentence (multiple-choice, yes/no questions whether they gate next steps or offer optional follow-up work, picking from a list, choosing between approaches), ask with the AskUserQuestion tool instead of plain text, so the user clicks an option instead of typing.

When presenting approaches, put the summary in each option's label and the pros/cons in its description. Plain text is fine when the answer is open-ended. This changes the format of questions, not whether to ask: never use it to ask permission for work you already have enough information to do.

## Workflow

- Before proposing a design of your own, invoke the best-practices skill to study prior art. Assume prior art exists; spend original design only where your problem actually differs
- When a finding invalidates the approach you're executing (contradicts it, or makes it unnecessary), stop and lead with it: what it kills, what the plan is now. Mentioning it in passing while continuing does not count

### Prefer Online Sources

Training data goes stale: library/framework/SDK APIs, config keys, CLI flags, cloud services, platform features, syntax, and versions change, and guessing has repeatedly cost debugging round-trips.

Invoke the find-docs skill BEFORE writing code or config that touches any of those, and BEFORE answering questions about them. Being about to write such code is trigger enough, even when no question was asked. Confidence is not an exemption, and neither is the library being well known. Answering from training data, or fetching a remembered docs URL instead of invoking the skill, does not satisfy this rule.

If the user provides URLs, WebFetch each one as a primary source before searching further. Never skip user-provided URLs. For topics find-docs covers poorly, WebFetch the official docs instead of falling back to training data.

Also see:

The Project CLAUDE.md

For project-specific instructions, put them in the project-level CLAUDE.md.

The highest-signal content in your project CLAUDE.md (or any skill) is the Gotchas section. Build these from the failure points Claude Code actually runs into.

Also see:

Per File Type Rules

For language-specific or per-file rules, put them in ~/.claude/rules/, so Claude Code only loads them when editing those file types.

For instance, ~/.claude/rules/python.md:

---
paths:
  - "**/*.py"
  - "**/pyproject.toml"
---

# Python

- When choosing a Python library or tool, check https://awesome-python.com/llms.txt before picking one
- Prefer the standard library over adding a dependency (supply-chain hardening) — tomllib over tomli, pathlib over external path libs
  - requests is fine since it's the de facto standard
- Version specifiers in pyproject.toml: use >= floors (uv's add-bounds default)
  - Exact reproducibility lives in uv.lock + uv sync --locked, so == pins in pyproject.toml would only duplicate the lockfile and block uv lock --upgrade
  - Pin exact == versions only where no lockfile exists (standalone scripts, requirements.txt)
- Use uv for project and environment management
  - uv run instead of python3 — picks up the project venv and dependencies automatically
  - Projects with a [build-system] need no-build = false in [tool.uv] — the global no-build = true in ~/.config/uv/uv.toml merges down and blocks the editable install
- Use pytest for testing
- Use ruff for linting and formatting
  - assert is fine in tests but use # noqa: S101 assert elsewhere
- When the linter flags something, read the rule it enforces (ruff rule <CODE>) and fix the code
  - Suppress with # noqa only when the rule does not apply to the project
  - All # noqa comments must include the rule name: # noqa: S603 subprocess-without-shell-equals-true or # noqa: S603 PLW1510 subprocess-without-shell-equals-true subprocess-run-without-check if multiple rules
- Use ty for type checking
- Use ty LSP tool for code navigation when grep's text matching would be ambiguous
- Use TypedDict for structured dicts — not plain dicts or dataclasses

The full rules I have:

Output Styles

Claude Code provides a built-in method to modify the system prompt to change how Claude responds: Output styles. For instance, mine forces caveman speak, so every reply drops articles, filler, and pleasantries while keeping every technical detail:

---
name: Caveman
description: Terse caveman-speak replies — full technical substance, no filler
keep-coding-instructions: true
---

Respond terse like smart caveman. All technical substance stay. Only fluff die.

Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/happy to), hedging, decorative tables/emoji. Fragments OK. Short synonyms (fix, not "implement a solution for"). Pattern: [thing] [action] [reason]. [next step].

Not: "Sure! I'd be happy to help. The issue is most likely caused by your auth middleware not validating token expiry."
Yes: "Bug in auth middleware. Token expiry check use < not <=. Fix:"

Never drop not/never/no/only/except — flipped meaning worse than any token saved. Numbers, units, technical terms exact.

Code blocks, commands, API names, error strings: byte-exact, never compressed.

Why an output style instead of the global CLAUDE.md? They land in different places: an output style becomes part of the system prompt, and Claude Code periodically reminds the model to stick to it mid-conversation, while CLAUDE.md gets injected as a user message, where it competes with all your other rules. Plus, you can switch styles in /config without touching your global rules.

One gotcha: output styles apply to the main conversation only — a subagent runs its own system prompt, so your style doesn't change how subagents respond.

Configurations

Settings

There are some useful configurations you could set in your ~/.claude/settings.json:

{
  "$schema": "https://json.schemastore.org/claude-code-settings.json",
  "env": {
    "CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY": "1",
    "CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1",
    "CLAUDE_CODE_RETRY_WATCHDOG": "1",
    "DISABLE_ERROR_REPORTING": "1",
    "DISABLE_EXTRA_USAGE_COMMAND": "1",
    "DISABLE_FEEDBACK_COMMAND": "1",
    "DISABLE_UPGRADE_COMMAND": "1"
  },
  "permissions": {
    "allow": ["..."],
    "deny": ["..."],
    "ask": ["..."],
    "defaultMode": "auto"
  },
  "model": "opus[1m]",
  "effortLevel": "high",
  "advisorModel": "fable",
  "cleanupPeriodDays": 365,
  "includeGitInstructions": false,
  "showClearContextOnPlanAccept": true,
  "teammateMode": "auto",
  "voice": { "enabled": true }
}

Highlights:

  • "CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1": Enable Agent Team feature, a fancy way to consume a huge amount of tokens
  • "permissions.defaultMode": "auto": Use this to pretend it's safer than --dangerously-skip-permissions
  • "advisorModel": "fable": Use something faster like sonnet as the main model, and let it consult fable when needed
  • "includeGitInstructions": false: Remove built-in commit/PR instructions and git status snapshot from the system prompt, since my commit skill covers that
  • "cleanupPeriodDays": 365: By default, your chat history (location: ~/.claude/projects/) will be deleted after 30 days
  • "voice": { "enabled": true }: Enable Voice Dictation feature. Code like a boss!

I turned all this settings-tweaking into a skill: audit-claude-settings fetches the latest settings and env-vars from official docs, diffs them against your actual config, and suggests changes tied to how you work.

The full settings I use:

Permissions

If you're not using a sandbox or devcontainer for Claude Code, you may want to block some evil commands in your ~/.claude/settings.json:

{
  "permissions": {
    "defaultMode": "auto",
    "deny": [
      "Read(~/.aws/**)",
      "Read(~/.config/**)",
      "Read(~/.docker/**)",
      "Read(~/.dropbox/**)",
      "Read(~/.gnupg/**)",
      "Read(~/.gsutil/**)",
      "Read(~/.kube/**)",
      "Read(~/.npmrc)",
      "Read(~/.orbstack/**)",
      "Read(~/.pypirc)",
      "Read(~/.ssh/**)",
      "Read(~/*_history)",
      "Read(~/**/*credential*)",
      "Read(~/Library/**)",
      "Edit(~/Library/**)",
      "Read(~/Dropbox/**)",
      "Edit(~/Dropbox/**)",
      "Read(//etc/**)",
      "Edit(//etc/**)",
      "Bash(git -c *)",
      "Bash(git --config-env*)",
      "Bash(git --git-dir*)",
      "Bash(gh repo delete *)",
      "Bash(su *)",
      "Bash(sudo *)",
      "Bash(passwd *)",
      "Bash(env *)",
      "Bash(printenv *)",
      "Bash(history *)",
      "Bash(fc *)",
      "Bash(eval *)",
      "Bash(exec *)",
      "Bash(rsync *)",
      "Bash(sftp *)",
      "Bash(telnet *)",
      "Bash(socat *)",
      "Bash(nc *)",
      "Bash(ncat *)",
      "Bash(netcat *)",
      "Bash(nmap *)",
      "Bash(kill *)",
      "Bash(killall *)",
      "Bash(pkill *)",
      "Bash(chmod *)",
      "Bash(chown *)",
      "Bash(chflags *)",
      "Bash(xattr *)",
      "Bash(diskutil *)",
      "Bash(mkfs *)",
      "Bash(security *)",
      "Bash(defaults *)",
      "Bash(launchctl *)",
      "Bash(osascript *)",
      "Bash(dscl *)",
      "Bash(networksetup *)",
      "Bash(scutil *)",
      "Bash(systemsetup *)",
      "Bash(pmset *)",
      "Bash(crontab *)"
    ],
    "ask": [
      "Bash(curl *)",
      "Bash(wget *)",
      "Bash(open *)",
      "Bash(* install *)",
      "Bash(bun add *)",
      "Bash(yarn add *)",
      "Bash(pnpm add *)",
      "Bash(uv add *)",
      "Bash(git push *)",
      "Bash(git remote add *)",
      "Bash(git remote rename *)",
      "Bash(git remote remove *)",
      "Bash(git remote rm *)",
      "Bash(git remote set-url *)",
      "Bash(git config remote.*)",
      "Bash(git config * remote.*)",
      "Bash(gh repo create *)",
      "Bash(gh repo rename *)",
      "Bash(npx supabase db *)"
    ]
  },
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          {
            "type": "command",
            "command": "python3 ~/.claude/hooks/guard-bash-paths.py"
          }
        ]
      }
    ]
  }
}

Rule precedence catches people out: Claude Code evaluates deny, then ask, then allow, and the first match wins — specificity never reorders them. So a narrow "allow": ["Bash(curl https://code.claude.com/docs/*)"] does nothing while "ask": ["Bash(curl *)"] is present, and a PreToolUse hook returning allow doesn't rescue it either, since a matching ask rule still prompts. Bash patterns have no negation, so carving an exception out of a broad ask rule means narrowing the ask rule itself.

However, "deny": ["Read(~/.aws/**)", "Read(~/.kube/**)", ...] alone is not enough, since Claude Code can still read sensitive files through the Bash tool. You can write a simple hook to intercept Bash commands that access blocked files, like this guard-bash-paths.py hook.

Though, Claude Code can still write a one-time script to read sensitive data and bypass all of the above defenses. So the safest approach is using sandbox after all.

Plugins

Claude Code Plugins are simply a way to package skills, commands, agents, hooks, MCP servers, LSP servers, and monitors. Distributing them as a plugin has the following advantages:

  • Auto update (versioned releases)
  • Auto hooks configuration (users don't need to edit their ~/.claude/settings.json manually)
  • Skills have a /plugin-name:your-skill-name prefix (no more conflicts)

To install a plugin, you need to add a marketplace first. A marketplace is usually just a GitHub repo. Think of it as a namespace.

/plugin marketplace add openai/codex-plugin-cc
/plugin marketplace add xai-org/grok-build-plugin-cc
/plugin marketplace add mattpocock/skills
/plugin marketplace add vinta/hal-9000

# browse plugins
/plugin

Recommended:

Skills

Skills can contain executable scripts and hooks, not just Markdown. Use with caution! When in doubt, have your agent review them first.

Here are skills I've used, mostly installed per project when needed:

# my skills
npx skills add https://github.com/vinta/hal-9000 --skill commit best-practices blindspot simple-english write-like-me refactor-agents-md -g

# workflow skills
npx skills add https://github.com/vinta/hal-9000 --skill commit best-practices blindspot -g
npx skills add https://github.com/mattpocock/skills -g

# writing skills
npx skills add https://github.com/vinta/hal-9000 --skill simple-english write-like-me
npx skills add https://github.com/shyuan/writing-humanizer

# doc skills
npx skills add https://github.com/upstash/context7 --skill find-docs -g
npx skills add https://github.com/humanlayer/skills --skill show-me -g

# language skills
npx skills add https://github.com/trailofbits/skills --skill modern-python
npx skills add https://github.com/trailofbits/skills-curated --skill python-code-simplifier
npx skills add https://github.com/dagster-io/skills --skill dignified-python
npx skills add https://github.com/wdm0006/python-skills

# backend skills
npx skills add https://github.com/vintasoftware/django-ai-plugins
npx skills add https://github.com/google/skills
npx skills add https://github.com/cloudflare/skills
npx skills add https://github.com/supabase/agent-skills
npx skills add https://github.com/planetscale/database-skills

# LLM API skills
npx skills add https://github.com/openai/skills
npx skills add https://github.com/google-gemini/gemini-skills

# frontend skills
npx skills add https://github.com/millionco/react-doctor
npx skills add https://github.com/vercel-labs/agent-skills
npx skills add https://github.com/vercel-labs/next-skills

# design skills
npx skills add https://github.com/openai/skills --skill frontend-skill
npx skills add https://github.com/pbakaus/impeccable
npx skills add https://github.com/nextlevelbuilder/ui-ux-pro-max-skill
npx skills add https://github.com/Leonxlnx/taste-skill

# seo/aeo skills
npx skills add https://github.com/warpdotdev/oz-skills --skill seo-aeo-audit

# video skills
npx skills add https://github.com/remotion-dev/skills

# browser skills
npx skills add https://github.com/microsoft/playwright-cli --skill playwright-cli -g

npx skills list -g
npx skills update -g
npx skills remove --all -g

Recommended:

  • /brainstorming from superpowers: When in doubt, start with this skill
  • /wayfinder from mattpocock: Let AI ask you a lot of questions
  • /find-docs from context7: Find the latest documentations
  • /frontend-design from impeccable: The better version of the official /frontend-design skill
  • /simplify: Run it often, you will like it

You can find more skills on skills.sh.

MCP Servers

You probably don't need any MCP servers if you can do the same thing with CLI + skills.

Context7 MCP

No, just use the ctx7 CLI with find-docs skill instead.

npx ctx7 setup

Playwright MCP

No, you should use the playwright-cli skill instead. Both tools support headed mode (the opposite of headless), if you'd like to see the browser.

npm install -g @playwright/cli
npx skills add https://github.com/microsoft/playwright-cli

GitHub MCP

No, you should use the gh command instead.

brew install gh

Codex MCP

Yes, ironically. Other coding agents like Claude Code can use Codex via MCP, which is slightly more stable than directly invoking it with codex exec via CLI.

# Codex reads your local .codex/config.toml by default
claude mcp add codex --scope user -- codex mcp-server

However, since OpenAI releases the official Claude Code plugin: codex-plugin-cc, you should probably use that instead.

Hooks

Both Claude Code and Codex support hooks. Hooks make Claude Code run specific commands on lifecycle events like SessionStart, UserPromptSubmit, and PreToolUse.

Instead of reminding Claude Code to run the linter or tests in your prompts (and it still forgets sometimes), just write a PostToolUse hook that runs deterministically:

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Write|Edit",
        "hooks": [
          {
            "type": "command",
            "command": "uv run ruff check . >&2 || exit 2",
            "if": "Edit(**/*.py)",
            "timeout": 30,
            "statusMessage": "Linting Python code..."
          }
        ]
      }
    ]
  }
}

It's worth noting that only exit code 2 blocks and feeds the output back to Claude, and only through stderr — while tools like ruff and ty print their diagnostics to stdout. A bare uv run ruff check . exits 1, a non-blocking error, so Claude only gets the first line of stderr, never the lint errors. >&2 moves them to the stream Claude reads, and || exit 2 makes the hook actually block.

Also, do the path matching in if, not inside a wrapper script. if takes the same permission rule syntax.

I also wrote some Claude Code plugins that use hooks:

For example, hal-session-auto-rename. Since Claude can message your other Claude Code sessions by name (you could also explicitly mention them with @session-name), a good session name actually matters. I found Claude Code already titles every session once, from its first real prompt, and stores it in the transcript, so I just wired that up to a UserPromptSubmit hook, which can set sessionTitle.

Useful Tips

Prompt Best Practices

Command Aliases

# in ~/.zshrc
alias cc="claude"
alias ccf="claude --model fable --effort max"
alias ccs="claude --model sonnet --effort high"
alias ccy="claude --dangerously-skip-permissions"
ccp() { claude --model sonnet --effort high --safe-mode --no-session-persistence --no-chrome -p "$*"; }

Use ccp for ad-hoc prompts:

ccp "commit"
ccp "list all .md in this repo"

Customize Your Statusline

Claude Code has a customizable statusline at the bottom of the terminal. You can run any script that outputs text.

Mine shows the current model, the current working folder, the git branch, and a grammar-corrected version of my last prompt (because my English needs all the help it can get). The grammar correction runs an ad-hoc claude command inside the statusline script.

Claude Code Statusline with English Grammar Check example

Run Ad-Hoc Claude Commands Inside Scripts

You can invoke claude as a one-shot CLI tool from hooks, statusline scripts, CI, or anywhere else. The trick is using the right flags to get a clean, isolated call with zero side effects:

cmd = """
    claude
    --model haiku
    --max-turns 1
    --setting-sources ""
    --tools ""
    --disable-slash-commands
    --no-session-persistence
    --no-chrome
    --safe-mode
    --print
"""

result = subprocess.run(
    [*shlex.split(cmd), your_prompt],
    capture_output=True,
    text=True,
    timeout=15,
    cwd="/tmp",
)

What each flag does:

  • --setting-sources "": don't load hooks (avoids infinite recursion if called from a hook)
  • --no-session-persistence and cwd="/tmp": avoid polluting your current context
  • --tools "": no file access, no bash, pure text in/out
  • --no-chrome: skip the Chrome integration
  • --safe-mode: skip CLAUDE.md/skills/plugins/MCP/auto-memory loading
Cloudflare Quick Tunnel (TryCloudflare)

Cloudflare Quick Tunnel (TryCloudflare)

Expose your local server to the Internet with one cloudflared command (just like ngrok). No account registration needed, no installation required (via docker run), and free.

# assume your local server is at http://localhost:3000
docker run --rm -it cloudflare/cloudflared tunnel --url http://localhost:3000

# if your local server is running inside a Docker container
docker run --rm -it cloudflare/cloudflared tunnel --url http://host.docker.internal:3000

ref:
https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/do-more-with-tunnels/trycloudflare/

You will see something like this in console:

+--------------------------------------------------------------------------------------------+
|  Your quick Tunnel has been created! Visit it at (it may take some time to be reachable):  |
|  https://YOUR_RANDOM_QUICK_TUNNEL_NAME.trycloudflare.com                                   |
+--------------------------------------------------------------------------------------------+

Then you're all set.

1Password CLI: How NOT to Store Plaintext AWS Credentials or .env on Localhost

1Password CLI: How NOT to Store Plaintext AWS Credentials or .env on Localhost

No More ~/.aws/credetials

According to AWS security best practices, human users should access AWS services using short-term credentials provided by IAM Identity Center. Long-term credentials ("Access Key ID" and "Secret Access Key") created by IAM users should be avoided, especially since they are often stored in plaintext on disk: ~/.aws/credetials.

However, if you somehow have to use AWS access keys but want an extra layer of protection, 1Password CLI can help.

ref:
https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html
https://developer.1password.com/docs/cli/get-started

First, delete your local plaintext AWS credentials. Don't worry, you could generate new one any time on AWS Management Console.

rm -rf ~/.aws/credetials

Re-create aws-cli configuration file, but DO NOT provide any credentials.

aws configure

AWS Access Key ID [None]: JUST PRESS ENTER, DO NOT TYPE ANYTHING
AWS Secret Access Key [None]: JUST PRESS ENTER, DO NOT TYPE ANYTHING
Default region name [None]: ap-northeast-1
Default output format [None]: json

Edit ~/.aws/credentials:

[your-profile-name]
credential_process = sh -c "op item get "AWS Access Key" --account=my.1password.com --vault=Private --format=json --fields label=AccessKeyId,label=SecretAccessKey | jq 'map({key: .label, value: .value}) | from_entries + {Version: 1}'"

The magic is credential_process which sourcing AWS credentials from an external process: 1Password CLI's op item get command.

The one-liner script assumes you have an item named AWS Access Key in a vault named Private in 1Password, and the item has following fields:

  • AccessKeyId
  • SecretAccessKey

ref:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html
https://developer.1password.com/docs/cli/reference/management-commands/item#item-get

That's it.

When you run aws-cli commands or access AWS services from your code via aws-sdk, your terminal will prompt you to unlock 1Password with biometrics to source AWS credentials (once per terminal session). No more plaintext AWS access keys on localhost!

# aws-cli
aws s3 ls --profile=perp
aws logs tail --profile=perp --region=ap-northeast-1 /aws/containerinsights/perp-staging/application --follow

# aws-sdk
AWS_PROFILE=perp OTHER_ENV=123 ts-node src/index.ts

# serverless v4 supports credential_process by default
# serverless v3 requires installing a plugin: serverless-better-credentials
# https://github.com/thomasmichaelwallace/serverless-better-credentials
sls deploy --stage=staging --aws-profile=perp

# if you're using serverless-offline, you might need to add the following configs to serverless.yml
custom:
  serverless-offline:
    useInProcess: true

It's worth noting that if you prefer not to use 1Password, there is also a tool called aws-vault which can achieve a similar goal.

ref:
https://github.com/99designs/aws-vault

No More .env

If you would like to store .env file entirely in 1Password, try 1Password Environments.

ref:
https://developer.1password.com/docs/environments
https://developer.1password.com/docs/environments/local-env-file

sysctl: Linux System Tweaking

sysctl: Linux System Tweaking

sysctl is a command-line tool to modify kernel parameters at runtime in Linux.

ref:
http://man7.org/linux/man-pages/man8/sysctl.8.html

Usage

List All Parameters

$ sudo sysctl -a
$ sudo sysctl -a | grep tcp

The parameters available are those listed under /proc/sys/.

$ cat /proc/sys/net/core/somaxconn
1024

Show the Entry of a Specified Parameter

$ sudo sysctl net.core.somaxconn
net.core.somaxconn = 1024

### Show the Value of a Specified Parameter

```console
$ sysctl -n net.core.somaxconn
1024

Change a Specified Parameter

# Elasticsearch
$ sysctl -w vm.max_map_count = 262143

# Redis
$ sysctl -w vm.overcommit_memory = 1

ref:
https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html
https://redis.io/topics/admin

Persistence

sysctl -w only modify parameters at runtime, and they would be set to default values after the system is restarted. You must write those settings in /etc/sysctl.conf to persist them.

# Do less swapping
vm.swappiness = 10
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2

# Prevents SYN DOS attacks. Applies to ipv6 as well, despite name.
net.ipv4.tcp_syncookies = 1

# Prevents ip spoofing.
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1

# Only groups within this id range can use ping.
net.ipv4.ping_group_range=999 59999

# Redirects can potentially be used to maliciously alter hosts routing tables.
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 1
net.ipv6.conf.all.accept_redirects = 0

# The source routing feature includes some known vulnerabilities.
net.ipv4.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0

# See RFC 1337
net.ipv4.tcp_rfc1337 = 1

# Enable IPv6 Privacy Extensions (see RFC4941 and RFC3041)
net.ipv6.conf.default.use_tempaddr = 2
net.ipv6.conf.all.use_tempaddr = 2

# Restarts computer after 120 seconds after kernel panic
kernel.panic = 120

# Users should not be able to create soft or hard links to files which they do not own. This mitigates several privilege escalation vulnerabilities.
fs.protected_hardlinks = 1
fs.protected_symlinks = 1

ref:
https://blog.runcloud.io/how-to-secure-your-linux-server/
https://www.percona.com/blog/2019/02/25/mysql-challenge-100k-connections/
https://www.nginx.com/blog/tuning-nginx/

Activate parameters from the configuration file.

$ sudo sysctl -p

Troubleshooting

OS error code 24: Too many open files

$ sudo vim /etc/sysctl.conf
fs.file-max = 601017

$ sudo sysctl -p

$ sudo vim /etc/security/limits.d/nofile.conf
* soft nofile 65535
* hard nofile 65535
root soft nofile 65535
root hard nofile 65535

$ ulimit -n 65535

OS error code 99: Cannot assign requested address

For MySQL. Because there's no available local network ports left. You might need to set net.ipv4.tcp_tw_reuse = 1 instead of net.ipv4.tcp_tw_recycle = 1.

$ sudo vim /etc/sysctl.conf
net.ipv4.tcp_tw_reuse = 1

$ sudo sysctl -p

ref:
https://www.percona.com/blog/2014/12/08/what-happens-when-your-application-cannot-open-yet-another-connection-to-mysql/
https://stackoverflow.com/questions/6426253/tcp-tw-reuse-vs-tcp-tw-recycle-which-to-use-or-both

Parameters are missing from sysctl -a or /proc/sys

Sometimes you might find some parameters are not in sysctl -a or /proc/sys.

You can find them in /sys:

$ echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
$ echo "never" > /sys/kernel/mm/transparent_hugepage/defrag

$ cat /sys/kernel/mm/transparent_hugepage/enabled

To persist them:

$ vim /etc/rc.local
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
   echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
   echo "never" > /sys/kernel/mm/transparent_hugepage/defrag
fi

$ systemctl enable rc-local

If /etc/rc.local doesn't exist, create one and run chmod 644 /etc/rc.local.

ref:
https://redis.io/topics/admin
https://unix.stackexchange.com/questions/99154/disable-transparent-hugepages