First commit! (TESTING)

This commit is contained in:
Amin.MasterkinG
2026-04-20 18:44:53 +03:30
commit df3c2c5c16
10 changed files with 2768 additions and 0 deletions
+35
View File
@@ -0,0 +1,35 @@
# Secrets & user config
config.json
.env
# CA certificates (generated at runtime, contain private keys)
ca/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
*.egg-info/
dist/
build/
*.egg
# Virtual environments
venv/
.venv/
env/
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# OS
.DS_Store
Thumbs.db
# Temp MITM certs
domainfront_certs_*/
+222
View File
@@ -0,0 +1,222 @@
# DomainFront Tunnel (MasterHttpRelayVPN Testing Python!)
A local HTTP proxy that bypasses DPI (Deep Packet Inspection) censorship using **domain fronting**. The proxy tunnels all browser traffic through a CDN — the TLS SNI shows an allowed domain (e.g. `www.google.com`) while the encrypted HTTP Host header routes to your relay endpoint.
## How It Works
```
Browser ──► Local Proxy ──► CDN (TLS SNI: www.google.com) ──► Your Relay ──► Target Website
DPI sees: "www.google.com" ✓
Actual destination: hidden
```
The DPI/firewall only sees the SNI field in the TLS handshake, which shows an innocuous, unblockable domain. The real destination is hidden inside the encrypted HTTP stream.
## Supported Modes
| Mode | Relay | Description |
|------|-------|-------------|
| `apps_script` | Google Apps Script | Fronts through `www.google.com``script.google.com`. Free, no server needed. |
| `google_fronting` | Google Cloud Run | Fronts through Google IP → your Cloud Run service. |
| `domain_fronting` | Cloudflare Worker | Classic domain fronting via Cloudflare CDN. |
| `custom_domain` | Custom domain on CF | Direct connection to your custom domain on Cloudflare. |
## Quick Start
### 1. Install
```bash
# Clone the repository
git clone https://github.com/masterking32/MasterHttpRelayVPN.git
cd domainfront-tunnel
# (Optional) Create a virtual environment
python -m venv venv
source venv/bin/activate # Linux/macOS
venv\Scripts\activate # Windows
# Install dependencies
pip install -r requirements.txt
```
> **Note:** Python 3.10+ is required. Core functionality has no external dependencies. The optional packages (`cryptography`, `h2`) enable MITM interception and HTTP/2 multiplexing for the `apps_script` mode.
### 2. Configure
```bash
cp config.example.json config.json
```
Edit `config.json` with your values:
```json
{
"mode": "apps_script",
"google_ip": "216.239.38.120",
"front_domain": "www.google.com",
"script_id": "YOUR_APPS_SCRIPT_DEPLOYMENT_ID",
"auth_key": "your-strong-secret-key",
"listen_host": "127.0.0.1",
"listen_port": 8085,
"log_level": "INFO",
"verify_ssl": true
}
```
### 3. Run
```bash
python main.py
```
### 4. Configure Your Browser
Set your browser's HTTP proxy to `127.0.0.1:8085` (or whatever `listen_host`:`listen_port` you configured).
For `apps_script` mode, you also need to install the generated CA certificate (`ca/ca.crt`) in your browser's trusted root CAs.
## Configuration Reference
### Required Fields
| Field | Description |
|-------|-------------|
| `mode` | One of: `apps_script`, `google_fronting`, `domain_fronting`, `custom_domain` |
| `auth_key` | Shared secret between the proxy and your relay endpoint |
### Mode-Specific Fields
| Field | Modes | Description |
|-------|-------|-------------|
| `script_id` | `apps_script` | Your deployed Apps Script ID (or array of IDs for load balancing) |
| `worker_host` | `domain_fronting`, `google_fronting` | Your Worker/Cloud Run hostname |
| `custom_domain` | `custom_domain` | Your custom domain on Cloudflare |
| `front_domain` | `domain_fronting`, `google_fronting`, `apps_script` | The domain shown in TLS SNI (default: `www.google.com`) |
| `google_ip` | `google_fronting`, `apps_script` | Google IP to connect to (default: `216.239.38.120`) |
### Optional Fields
| Field | Default | Description |
|-------|---------|-------------|
| `listen_host` | `127.0.0.1` | Local proxy bind address |
| `listen_port` | `8080` | Local proxy port |
| `log_level` | `INFO` | Logging level: `DEBUG`, `INFO`, `WARNING`, `ERROR` |
| `verify_ssl` | `true` | Verify TLS certificates |
| `worker_path` | `""` | URL path prefix for the worker |
| `script_ids` | — | Array of Apps Script IDs for round-robin load balancing |
## Environment Variables
All settings can be overridden via environment variables (useful for containers/CI):
| Variable | Overrides |
|----------|-----------|
| `DFT_CONFIG` | Config file path |
| `DFT_AUTH_KEY` | `auth_key` |
| `DFT_SCRIPT_ID` | `script_id` |
| `DFT_PORT` | `listen_port` |
| `DFT_HOST` | `listen_host` |
| `DFT_LOG_LEVEL` | `log_level` |
## CLI Usage
```
usage: domainfront-tunnel [-h] [-c CONFIG] [-p PORT] [--host HOST]
[--log-level {DEBUG,INFO,WARNING,ERROR}] [-v]
options:
-c, --config CONFIG Path to config file (default: config.json)
-p, --port PORT Override listen port
--host HOST Override listen host
--log-level LEVEL Override log level
-v, --version Show version and exit
```
### Examples
```bash
# Basic usage
python main.py
# Custom config file
python main.py -c /path/to/my-config.json
# Override port
python main.py -p 9090
# Debug logging
python main.py --log-level DEBUG
# Using environment variables
DFT_AUTH_KEY=my-secret DFT_PORT=9090 python main.py
```
## Apps Script Setup
1. Go to [Google Apps Script](https://script.google.com/) and create a new project.
2. Paste your relay script code into `Code.gs`.
3. Deploy as a **Web App**:
- Execute as: **Me**
- Who has access: **Anyone**
4. Copy the **Deployment ID** and paste it into `config.json` as `script_id`.
5. Set a strong `auth_key` in both the Apps Script and `config.json`.
### Multiple Script IDs (Load Balancing)
For higher throughput, deploy multiple copies and use an array:
```json
{
"script_ids": [
"DEPLOYMENT_ID_1",
"DEPLOYMENT_ID_2",
"DEPLOYMENT_ID_3"
]
}
```
## Architecture
```
┌─────────┐ ┌──────────────┐ ┌─────────────┐ ┌──────────┐
│ Browser │────►│ Local Proxy │────►│ CDN / Google │────►│ Relay │──► Internet
│ │◄────│ (this tool) │◄────│ (fronted) │◄────│ Endpoint │◄──
└─────────┘ └──────────────┘ └─────────────┘ └──────────┘
HTTP/CONNECT TLS (SNI: ok) Fetch target
MITM (optional) Host: relay Return response
```
### Key Components
| File | Purpose |
|------|---------|
| `main.py` | Entry point, config loading, CLI |
| `proxy_server.py` | Local HTTP/CONNECT proxy server |
| `domain_fronter.py` | Domain fronting engine, connection pooling, relay logic |
| `h2_transport.py` | HTTP/2 multiplexed transport (optional, for performance) |
| `mitm.py` | MITM certificate manager for HTTPS interception |
| `ws.py` | WebSocket frame encoder/decoder (RFC 6455) |
## Performance Features
- **HTTP/2 multiplexing**: Single TLS connection handles 100+ concurrent requests
- **Connection pooling**: Pre-warmed TLS connection pool with automatic maintenance
- **Request batching**: Groups concurrent requests into single relay calls
- **Request coalescing**: Deduplicates identical concurrent GET requests
- **Parallel range downloads**: Splits large downloads into concurrent chunks
- **Response caching**: LRU cache for static assets (configurable, 50 MB default)
## Security Notes
- **Never commit `config.json`** — it contains your `auth_key`. The `.gitignore` excludes it.
- The `ca/` directory contains your generated CA private key. Keep it secure.
- Use a strong, unique `auth_key` to prevent unauthorized use of your relay.
- Set `listen_host` to `127.0.0.1` (not `0.0.0.0`) unless you need LAN access.
## Special Thanks
Special thanks to [@abolix](https://github.com/abolix) for making this project possible.
## License
MIT
+12
View File
@@ -0,0 +1,12 @@
{
"_comment": "Copy this file to config.json and fill in your values",
"mode": "apps_script",
"google_ip": "216.239.38.120",
"front_domain": "www.google.com",
"script_id": "YOUR_APPS_SCRIPT_DEPLOYMENT_ID",
"auth_key": "CHANGE_ME_TO_A_STRONG_SECRET",
"listen_host": "127.0.0.1",
"listen_port": 8085,
"log_level": "INFO",
"verify_ssl": true
}
+1159
View File
File diff suppressed because it is too large Load Diff
+419
View File
@@ -0,0 +1,419 @@
"""
HTTP/2 multiplexed transport for domain-fronted connections.
One TLS connection → many concurrent HTTP/2 streams → massive throughput.
Eliminates per-request TLS handshake overhead entirely.
Instead of a pool of 30 HTTP/1.1 connections (each handling 1 request),
this uses a SINGLE HTTP/2 connection handling 100+ concurrent requests.
Performance comparison:
HTTP/1.1 pool: 30 connections × 1 request = 30 concurrent requests max
HTTP/2 mux: 1 connection × 100 streams = 100 concurrent requests
Requires: pip install h2
"""
import asyncio
import gzip
import logging
import socket
import ssl
from urllib.parse import urlparse
log = logging.getLogger("H2")
try:
import h2.connection
import h2.config
import h2.events
import h2.settings
H2_AVAILABLE = True
except ImportError:
H2_AVAILABLE = False
class _StreamState:
"""State for a single in-flight HTTP/2 stream."""
__slots__ = ("status", "headers", "data", "done", "error")
def __init__(self):
self.status = 0
self.headers: dict[str, str] = {}
self.data = bytearray()
self.done = asyncio.Event()
self.error: str | None = None
class H2Transport:
"""
Persistent HTTP/2 connection with automatic stream multiplexing.
All relay requests share ONE TLS connection. Each request becomes
an independent HTTP/2 stream, running fully concurrently.
Features:
- Auto-connect on first use
- Auto-reconnect on connection loss
- Redirect following (as new streams, same connection)
- Gzip decompression
- Configurable max concurrency
"""
def __init__(self, connect_host: str, sni_host: str,
verify_ssl: bool = True):
self.connect_host = connect_host
self.sni_host = sni_host
self.verify_ssl = verify_ssl
self._reader: asyncio.StreamReader | None = None
self._writer: asyncio.StreamWriter | None = None
self._h2: "h2.connection.H2Connection | None" = None
self._connected = False
self._write_lock = asyncio.Lock()
self._connect_lock = asyncio.Lock()
self._read_task: asyncio.Task | None = None
# Per-stream tracking
self._streams: dict[int, _StreamState] = {}
# Stats
self.total_requests = 0
self.total_streams = 0
# ── Connection lifecycle ──────────────────────────────────────
@property
def is_connected(self) -> bool:
return self._connected
async def ensure_connected(self):
"""Connect if not already connected."""
if self._connected:
return
async with self._connect_lock:
if self._connected:
return
await self._do_connect()
async def _do_connect(self):
"""Establish the HTTP/2 connection with optimized socket settings."""
ctx = ssl.create_default_context()
# Advertise both h2 and http/1.1 — some DPI blocks h2-only ALPN
ctx.set_alpn_protocols(["h2", "http/1.1"])
if not self.verify_ssl:
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
# Create raw TCP socket with TCP_NODELAY BEFORE TLS handshake.
# Nagle's algorithm can delay small writes (H2 frames) by up to 200ms
# waiting to coalesce — TCP_NODELAY forces immediate send.
raw = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
raw.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
raw.setblocking(False)
try:
await asyncio.wait_for(
asyncio.get_event_loop().sock_connect(
raw, (self.connect_host, 443)
),
timeout=15,
)
self._reader, self._writer = await asyncio.wait_for(
asyncio.open_connection(
ssl=ctx,
server_hostname=self.sni_host,
sock=raw,
),
timeout=15,
)
except Exception:
raw.close()
raise
# Verify we actually got HTTP/2
ssl_obj = self._writer.get_extra_info("ssl_object")
negotiated = ssl_obj.selected_alpn_protocol() if ssl_obj else None
if negotiated != "h2":
self._writer.close()
raise RuntimeError(
f"H2 ALPN negotiation failed (got {negotiated!r})"
)
config = h2.config.H2Configuration(
client_side=True,
header_encoding="utf-8",
)
self._h2 = h2.connection.H2Connection(config=config)
self._h2.initiate_connection()
# Connection-level flow control: ~16MB window
self._h2.increment_flow_control_window(2 ** 24 - 65535)
# Per-stream settings: 1MB initial window, disable server push
self._h2.update_settings({
h2.settings.SettingCodes.INITIAL_WINDOW_SIZE: 1 * 1024 * 1024,
h2.settings.SettingCodes.ENABLE_PUSH: 0,
})
await self._flush()
self._connected = True
self._read_task = asyncio.create_task(self._reader_loop())
log.info("H2 connected → %s (SNI=%s, TCP_NODELAY=on)",
self.connect_host, self.sni_host)
async def reconnect(self):
"""Close current connection and re-establish."""
await self._close_internal()
await self._do_connect()
async def _close_internal(self):
self._connected = False
if self._read_task:
self._read_task.cancel()
self._read_task = None
if self._writer:
try:
self._writer.close()
except Exception:
pass
self._writer = None
# Wake all pending streams so they can raise
for state in self._streams.values():
state.error = "Connection closed"
state.done.set()
self._streams.clear()
# ── Public API ────────────────────────────────────────────────
async def request(self, method: str, path: str, host: str,
headers: dict | None = None,
body: bytes | None = None,
timeout: float = 25,
follow_redirects: int = 5) -> tuple[int, dict, bytes]:
"""
Send an HTTP/2 request and return (status, headers, body).
Thread-safe: many concurrent calls each get their own stream.
Redirects are followed as new streams on the same connection.
"""
await self.ensure_connected()
self.total_requests += 1
for _ in range(follow_redirects + 1):
status, resp_headers, resp_body = await self._single_request(
method, path, host, headers, body, timeout,
)
if status not in (301, 302, 303, 307, 308):
return status, resp_headers, resp_body
location = resp_headers.get("location", "")
if not location:
return status, resp_headers, resp_body
parsed = urlparse(location)
path = parsed.path + ("?" + parsed.query if parsed.query else "")
host = parsed.netloc or host
method = "GET"
body = None
headers = None # Drop request headers on redirect
return status, resp_headers, resp_body
# ── Stream handling ───────────────────────────────────────────
async def _single_request(self, method, path, host, headers, body,
timeout) -> tuple[int, dict, bytes]:
"""Send one HTTP/2 request on a new stream, wait for response."""
if not self._connected:
await self.ensure_connected()
stream_id = None
async with self._write_lock:
try:
stream_id = self._h2.get_next_available_stream_id()
except Exception:
# Connection is stale — reconnect
await self.reconnect()
stream_id = self._h2.get_next_available_stream_id()
h2_headers = [
(":method", method),
(":path", path),
(":authority", host),
(":scheme", "https"),
("accept-encoding", "gzip"),
]
if headers:
for k, v in headers.items():
h2_headers.append((k.lower(), str(v)))
end_stream = not body
self._h2.send_headers(stream_id, h2_headers, end_stream=end_stream)
if body:
# Send body (may need chunking for flow control)
self._send_body(stream_id, body)
state = _StreamState()
self._streams[stream_id] = state
self.total_streams += 1
await self._flush()
# Wait for complete response
try:
await asyncio.wait_for(state.done.wait(), timeout=timeout)
except asyncio.TimeoutError:
self._streams.pop(stream_id, None)
raise TimeoutError(
f"H2 stream {stream_id} timed out ({timeout}s)"
)
self._streams.pop(stream_id, None)
if state.error:
raise ConnectionError(f"H2 stream error: {state.error}")
# Auto-decompress gzip
resp_body = bytes(state.data)
if state.headers.get("content-encoding", "").lower() == "gzip":
try:
resp_body = gzip.decompress(resp_body)
except Exception:
pass
return state.status, state.headers, resp_body
def _send_body(self, stream_id: int, body: bytes):
"""Send request body, respecting H2 flow control window."""
# For small bodies (typical JSON payloads), send in one shot
while body:
max_size = self._h2.local_settings.max_frame_size
window = self._h2.local_flow_control_window(stream_id)
send_size = min(len(body), max_size, window)
if send_size <= 0:
# Flow control full — let the reader loop process
# window updates before we continue
break
end = send_size >= len(body)
self._h2.send_data(stream_id, body[:send_size], end_stream=end)
body = body[send_size:]
# ── Background reader ─────────────────────────────────────────
async def _reader_loop(self):
"""Background: read H2 frames, dispatch events to waiting streams."""
try:
while self._connected:
data = await self._reader.read(65536)
if not data:
log.warning("H2 remote closed connection")
break
try:
events = self._h2.receive_data(data)
except Exception as e:
log.error("H2 protocol error: %s", e)
break
for event in events:
self._dispatch(event)
# Send pending data (acks, window updates, ping responses)
async with self._write_lock:
await self._flush()
except asyncio.CancelledError:
pass
except Exception as e:
log.error("H2 reader error: %s", e)
finally:
self._connected = False
for state in self._streams.values():
if not state.done.is_set():
state.error = "Connection lost"
state.done.set()
log.info("H2 reader loop ended")
def _dispatch(self, event):
"""Route a single h2 event to its stream."""
if isinstance(event, h2.events.ResponseReceived):
state = self._streams.get(event.stream_id)
if state:
for name, value in event.headers:
n = name if isinstance(name, str) else name.decode()
v = value if isinstance(value, str) else value.decode()
if n == ":status":
state.status = int(v)
else:
state.headers[n] = v
elif isinstance(event, h2.events.DataReceived):
state = self._streams.get(event.stream_id)
if state:
state.data.extend(event.data)
# Always acknowledge received data for flow control
self._h2.acknowledge_received_data(
event.flow_controlled_length, event.stream_id
)
elif isinstance(event, h2.events.StreamEnded):
state = self._streams.get(event.stream_id)
if state:
state.done.set()
elif isinstance(event, h2.events.StreamReset):
state = self._streams.get(event.stream_id)
if state:
state.error = f"Stream reset (code={event.error_code})"
state.done.set()
elif isinstance(event, h2.events.WindowUpdated):
pass # h2 library handles window bookkeeping
elif isinstance(event, h2.events.SettingsAcknowledged):
pass
elif isinstance(event, h2.events.PingReceived):
pass # h2 library auto-responds
elif isinstance(event, h2.events.PingAckReceived):
pass # keepalive confirmed
# ── Internal ──────────────────────────────────────────────────
async def _flush(self):
"""Write pending H2 frame data to the socket."""
data = self._h2.data_to_send()
if data and self._writer:
self._writer.write(data)
await self._writer.drain()
async def close(self):
"""Gracefully close the HTTP/2 connection."""
if self._h2 and self._connected:
try:
self._h2.close_connection()
async with self._write_lock:
await self._flush()
except Exception:
pass
await self._close_internal()
async def ping(self):
"""Send an H2 PING frame to keep the connection alive."""
if not self._connected or not self._h2:
return
try:
async with self._write_lock:
if not self._connected:
return
self._h2.ping(b"\x00" * 8)
await self._flush()
except Exception as e:
log.debug("H2 PING failed: %s", e)
+164
View File
@@ -0,0 +1,164 @@
#!/usr/bin/env python3
"""
DomainFront Tunnel — Bypass DPI censorship via Domain Fronting.
Run a local HTTP proxy that tunnels all traffic through a CDN using
domain fronting: the TLS SNI shows an allowed domain while the encrypted
HTTP Host header routes to your Cloudflare Worker relay.
"""
import argparse
import asyncio
import json
import logging
import os
import sys
from proxy_server import ProxyServer
__version__ = "1.0.0"
def setup_logging(level_name: str):
level = getattr(logging, level_name.upper(), logging.INFO)
logging.basicConfig(
level=level,
format="%(asctime)s [%(name)-12s] %(levelname)-7s %(message)s",
datefmt="%H:%M:%S",
)
def parse_args():
parser = argparse.ArgumentParser(
prog="domainfront-tunnel",
description="Local HTTP proxy that tunnels traffic through domain fronting.",
)
parser.add_argument(
"-c", "--config",
default=os.environ.get("DFT_CONFIG", "config.json"),
help="Path to config file (default: config.json, env: DFT_CONFIG)",
)
parser.add_argument(
"-p", "--port",
type=int,
default=None,
help="Override listen port (env: DFT_PORT)",
)
parser.add_argument(
"--host",
default=None,
help="Override listen host (env: DFT_HOST)",
)
parser.add_argument(
"--log-level",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
default=None,
help="Override log level (env: DFT_LOG_LEVEL)",
)
parser.add_argument(
"-v", "--version",
action="version",
version=f"%(prog)s {__version__}",
)
return parser.parse_args()
def main():
args = parse_args()
config_path = args.config
try:
with open(config_path) as f:
config = json.load(f)
except FileNotFoundError:
print(f"Config not found: {config_path}")
print("Copy config.example.json to config.json and fill in your values.")
sys.exit(1)
except json.JSONDecodeError as e:
print(f"Invalid JSON in config: {e}")
sys.exit(1)
# Environment variable overrides
if os.environ.get("DFT_AUTH_KEY"):
config["auth_key"] = os.environ["DFT_AUTH_KEY"]
if os.environ.get("DFT_SCRIPT_ID"):
config["script_id"] = os.environ["DFT_SCRIPT_ID"]
# CLI argument overrides
if args.port is not None:
config["listen_port"] = args.port
elif os.environ.get("DFT_PORT"):
config["listen_port"] = int(os.environ["DFT_PORT"])
if args.host is not None:
config["listen_host"] = args.host
elif os.environ.get("DFT_HOST"):
config["listen_host"] = os.environ["DFT_HOST"]
if args.log_level is not None:
config["log_level"] = args.log_level
elif os.environ.get("DFT_LOG_LEVEL"):
config["log_level"] = os.environ["DFT_LOG_LEVEL"]
for key in ("auth_key",):
if key not in config:
print(f"Missing required config key: {key}")
sys.exit(1)
mode = config.get("mode", "domain_fronting")
if mode == "custom_domain" and "custom_domain" not in config:
print("Mode 'custom_domain' requires 'custom_domain' in config")
sys.exit(1)
if mode == "domain_fronting":
for key in ("front_domain", "worker_host"):
if key not in config:
print(f"Mode 'domain_fronting' requires '{key}' in config")
sys.exit(1)
if mode == "google_fronting":
if "worker_host" not in config:
print("Mode 'google_fronting' requires 'worker_host' in config (your Cloud Run URL)")
sys.exit(1)
if mode == "apps_script":
sid = config.get("script_ids") or config.get("script_id")
if not sid or (isinstance(sid, str) and sid == "YOUR_APPS_SCRIPT_DEPLOYMENT_ID"):
print("Mode 'apps_script' requires 'script_id' in config.")
print("Deploy the Apps Script from appsscript/Code.gs and paste the Deployment ID.")
sys.exit(1)
setup_logging(config.get("log_level", "INFO"))
log = logging.getLogger("Main")
mode = config.get("mode", "domain_fronting")
log.info("DomainFront Tunnel starting (mode: %s)", mode)
if mode == "custom_domain":
log.info("Custom domain : %s", config["custom_domain"])
elif mode == "google_fronting":
log.info("Google fronting : SNI=%s → Host=%s",
config.get("front_domain", "www.google.com"), config["worker_host"])
log.info("Google IP : %s", config.get("google_ip", "216.239.38.120"))
elif mode == "apps_script":
log.info("Apps Script relay : SNI=%s → script.google.com",
config.get("front_domain", "www.google.com"))
script_ids = config.get("script_ids") or config.get("script_id")
if isinstance(script_ids, list):
log.info("Script IDs : %d scripts (round-robin)", len(script_ids))
for i, sid in enumerate(script_ids):
log.info(" [%d] %s", i + 1, sid)
else:
log.info("Script ID : %s", script_ids)
log.info("MITM enabled — install ca/ca.crt in your browser!")
else:
log.info("Front domain (SNI) : %s", config.get("front_domain", "?"))
log.info("Worker host (Host) : %s", config.get("worker_host", "?"))
log.info("Proxy address : %s:%d", config.get("listen_host", "127.0.0.1"), config.get("listen_port", 8080))
try:
asyncio.run(ProxyServer(config).start())
except KeyboardInterrupt:
log.info("Stopped")
if __name__ == "__main__":
main()
+153
View File
@@ -0,0 +1,153 @@
"""
MITM certificate manager for HTTPS interception.
Generates a CA certificate (once, stored as files) and per-domain
certificates (on the fly, cached in memory) so the local proxy can
decrypt HTTPS traffic and relay it through Apps Script.
The user must install ca/ca.crt in their browser's trusted CAs once.
Requires: pip install cryptography
"""
import datetime
import logging
import os
import ssl
import tempfile
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.x509.oid import NameOID
log = logging.getLogger("MITM")
CA_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "ca")
CA_KEY_FILE = os.path.join(CA_DIR, "ca.key")
CA_CERT_FILE = os.path.join(CA_DIR, "ca.crt")
class MITMCertManager:
def __init__(self):
self._ca_key = None
self._ca_cert = None
self._ctx_cache: dict[str, ssl.SSLContext] = {}
self._cert_dir = tempfile.mkdtemp(prefix="domainfront_certs_")
self._ensure_ca()
def _ensure_ca(self):
if os.path.exists(CA_KEY_FILE) and os.path.exists(CA_CERT_FILE):
with open(CA_KEY_FILE, "rb") as f:
self._ca_key = serialization.load_pem_private_key(
f.read(), password=None
)
with open(CA_CERT_FILE, "rb") as f:
self._ca_cert = x509.load_pem_x509_certificate(f.read())
log.info("Loaded CA from %s", CA_DIR)
else:
self._create_ca()
def _create_ca(self):
os.makedirs(CA_DIR, exist_ok=True)
self._ca_key = rsa.generate_private_key(
public_exponent=65537, key_size=2048
)
subject = issuer = x509.Name([
x509.NameAttribute(NameOID.COMMON_NAME, "DomainFront Tunnel CA"),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "DomainFront Tunnel"),
])
now = datetime.datetime.now(datetime.timezone.utc)
self._ca_cert = (
x509.CertificateBuilder()
.subject_name(subject)
.issuer_name(issuer)
.public_key(self._ca_key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now)
.not_valid_after(now + datetime.timedelta(days=3650))
.add_extension(
x509.BasicConstraints(ca=True, path_length=0), critical=True
)
.add_extension(
x509.KeyUsage(
digital_signature=True,
key_cert_sign=True,
crl_sign=True,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
encipher_only=False,
decipher_only=False,
),
critical=True,
)
.sign(self._ca_key, hashes.SHA256())
)
with open(CA_KEY_FILE, "wb") as f:
f.write(
self._ca_key.private_bytes(
serialization.Encoding.PEM,
serialization.PrivateFormat.TraditionalOpenSSL,
serialization.NoEncryption(),
)
)
with open(CA_CERT_FILE, "wb") as f:
f.write(self._ca_cert.public_bytes(serialization.Encoding.PEM))
log.warning("Generated new CA certificate: %s", CA_CERT_FILE)
log.warning(">>> Install this file in your browser's Trusted Root CAs! <<<")
def get_server_context(self, domain: str) -> ssl.SSLContext:
if domain not in self._ctx_cache:
key_pem, cert_pem = self._generate_domain_cert(domain)
cert_file = os.path.join(self._cert_dir, f"{domain}.crt")
key_file = os.path.join(self._cert_dir, f"{domain}.key")
ca_pem = self._ca_cert.public_bytes(serialization.Encoding.PEM)
with open(cert_file, "wb") as f:
f.write(cert_pem + ca_pem)
with open(key_file, "wb") as f:
f.write(key_pem)
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.set_alpn_protocols(["http/1.1"])
ctx.load_cert_chain(cert_file, key_file)
self._ctx_cache[domain] = ctx
return self._ctx_cache[domain]
def _generate_domain_cert(self, domain: str):
key = rsa.generate_private_key(
public_exponent=65537, key_size=2048
)
subject = x509.Name([
x509.NameAttribute(NameOID.COMMON_NAME, domain),
])
now = datetime.datetime.now(datetime.timezone.utc)
cert = (
x509.CertificateBuilder()
.subject_name(subject)
.issuer_name(self._ca_cert.subject)
.public_key(key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now)
.not_valid_after(now + datetime.timedelta(days=365))
.add_extension(
x509.SubjectAlternativeName([x509.DNSName(domain)]),
critical=False,
)
.sign(self._ca_key, hashes.SHA256())
)
key_pem = key.private_bytes(
serialization.Encoding.PEM,
serialization.PrivateFormat.TraditionalOpenSSL,
serialization.NoEncryption(),
)
cert_pem = cert.public_bytes(serialization.Encoding.PEM)
return key_pem, cert_pem
+520
View File
@@ -0,0 +1,520 @@
"""
Local HTTP proxy server.
Intercepts the user's browser traffic and forwards everything through
a domain-fronted connection to a CDN worker or Apps Script relay.
Supports:
- CONNECT method → WebSocket tunnel (modes 1-3) or MITM relay (apps_script)
- GET / POST etc. → HTTP forwarding (modes 1-3) or JSON relay (apps_script)
"""
import asyncio
import logging
import re
import time
from domain_fronter import DomainFronter
log = logging.getLogger("Proxy")
class ResponseCache:
"""Simple LRU response cache — avoids repeated relay calls."""
def __init__(self, max_mb: int = 50):
self._store: dict[str, tuple[bytes, float]] = {}
self._size = 0
self._max = max_mb * 1024 * 1024
self.hits = 0
self.misses = 0
def get(self, url: str) -> bytes | None:
entry = self._store.get(url)
if not entry:
self.misses += 1
return None
raw, expires = entry
if time.time() > expires:
self._size -= len(raw)
del self._store[url]
self.misses += 1
return None
self.hits += 1
return raw
def put(self, url: str, raw_response: bytes, ttl: int = 300):
size = len(raw_response)
if size > self._max // 4 or size == 0:
return
# Evict oldest to make room
while self._size + size > self._max and self._store:
oldest = next(iter(self._store))
self._size -= len(self._store[oldest][0])
del self._store[oldest]
if url in self._store:
self._size -= len(self._store[url][0])
self._store[url] = (raw_response, time.time() + ttl)
self._size += size
@staticmethod
def parse_ttl(raw_response: bytes, url: str) -> int:
"""Determine cache TTL from response headers and URL."""
hdr_end = raw_response.find(b"\r\n\r\n")
if hdr_end < 0:
return 0
hdr = raw_response[:hdr_end].decode(errors="replace").lower()
# Don't cache errors or non-200
if b"HTTP/1.1 200" not in raw_response[:20]:
return 0
if "no-store" in hdr:
return 0
# Explicit max-age
m = re.search(r"max-age=(\d+)", hdr)
if m:
return min(int(m.group(1)), 86400)
# Heuristic by content type / extension
path = url.split("?")[0].lower()
static_exts = (
".css", ".js", ".woff", ".woff2", ".ttf", ".eot",
".png", ".jpg", ".jpeg", ".gif", ".webp", ".svg", ".ico",
".mp3", ".mp4", ".wasm",
)
for ext in static_exts:
if path.endswith(ext):
return 3600 # 1 hour for static assets
ct_m = re.search(r"content-type:\s*([^\r\n]+)", hdr)
ct = ct_m.group(1) if ct_m else ""
if "image/" in ct or "font/" in ct:
return 3600
if "text/css" in ct or "javascript" in ct:
return 1800
if "text/html" in ct or "application/json" in ct:
return 0 # don't cache dynamic content by default
return 0
class ProxyServer:
def __init__(self, config: dict):
self.host = config.get("listen_host", "127.0.0.1")
self.port = config.get("listen_port", 8080)
self.mode = config.get("mode", "domain_fronting")
self.fronter = DomainFronter(config)
self.mitm = None
self._cache = ResponseCache(max_mb=50)
# Persistent HTTP tunnel cache for google_fronting mode
# Key: "host:port" → (tunnel_reader, tunnel_writer, lock)
self._http_tunnels: dict = {}
self._tunnel_lock = asyncio.Lock()
if self.mode == "apps_script":
try:
from mitm import MITMCertManager
self.mitm = MITMCertManager()
except ImportError:
log.error("apps_script mode requires 'cryptography' package.")
log.error("Run: pip install cryptography")
raise SystemExit(1)
async def start(self):
srv = await asyncio.start_server(self._on_client, self.host, self.port)
log.info(
"Listening on %s:%d — configure your browser HTTP proxy to this address",
self.host, self.port,
)
async with srv:
await srv.serve_forever()
# ── client handler ────────────────────────────────────────────
async def _on_client(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter):
addr = writer.get_extra_info("peername")
try:
first_line = await asyncio.wait_for(reader.readline(), timeout=30)
if not first_line:
return
# Read remaining headers
header_block = first_line
while True:
line = await asyncio.wait_for(reader.readline(), timeout=10)
header_block += line
if line in (b"\r\n", b"\n", b""):
break
request_line = first_line.decode(errors="replace").strip()
parts = request_line.split(" ", 2)
if len(parts) < 2:
return
method = parts[0].upper()
if method == "CONNECT":
await self._do_connect(parts[1], reader, writer)
else:
await self._do_http(header_block, reader, writer)
except asyncio.TimeoutError:
log.debug("Timeout: %s", addr)
except Exception as e:
log.error("Error (%s): %s", addr, e)
finally:
try:
writer.close()
await writer.wait_closed()
except Exception:
pass
# ── CONNECT (HTTPS tunnelling) ────────────────────────────────
async def _do_connect(self, target: str, reader, writer):
host, _, port = target.rpartition(":")
port = int(port) if port else 443
if not host:
host, port = target, 443
log.info("CONNECT → %s:%d", host, port)
writer.write(b"HTTP/1.1 200 Connection Established\r\n\r\n")
await writer.drain()
if self.mode == "apps_script":
# Google services: tunnel directly (no MITM) to avoid
# Google's anti-bot detection from Apps Script IPs/UA.
if self._is_google_domain(host):
log.info("Direct tunnel → %s (Google domain, skipping relay)", host)
await self._do_direct_tunnel(host, port, reader, writer)
else:
await self._do_mitm_connect(host, port, reader, writer)
else:
await self.fronter.tunnel(host, port, reader, writer)
# ── Google domain detection ───────────────────────────────────
# Only domains whose SNI the ISP does NOT block.
# YouTube/googlevideo are blocked by SNI inspection in Iran,
# so they MUST go through the MITM relay (domain-fronted).
_GOOGLE_SUFFIXES = (
".google.com", ".google.co",
".googleapis.com", ".gstatic.com",
".googleusercontent.com",
)
_GOOGLE_EXACT = {
"google.com", "gstatic.com", "googleapis.com",
}
def _is_google_domain(self, host: str) -> bool:
"""Return True if host is a Google-owned domain."""
h = host.lower().rstrip(".")
if h in self._GOOGLE_EXACT:
return True
for suffix in self._GOOGLE_SUFFIXES:
if h.endswith(suffix):
return True
return False
# ── Direct tunnel (no MITM) ───────────────────────────────────
async def _do_direct_tunnel(self, host: str, port: int,
reader: asyncio.StreamReader,
writer: asyncio.StreamWriter):
"""Pipe raw TLS bytes directly to the target server.
Used for Google domains: the browser's TLS goes end-to-end
with Google, preserving real User-Agent and avoiding
Apps Script IP/bot-detection issues.
"""
google_ip = self.fronter.connect_host
try:
r_remote, w_remote = await asyncio.wait_for(
asyncio.open_connection(google_ip, port), timeout=10
)
except Exception as e:
log.error("Direct tunnel connect failed (%s via %s): %s",
host, google_ip, e)
return
async def pipe(src, dst, label):
try:
while True:
data = await src.read(65536)
if not data:
break
dst.write(data)
await dst.drain()
except (ConnectionError, asyncio.CancelledError):
pass
except Exception as e:
log.debug("Pipe %s ended: %s", label, e)
finally:
try:
dst.close()
except Exception:
pass
await asyncio.gather(
pipe(reader, w_remote, f"client→{host}"),
pipe(r_remote, writer, f"{host}→client"),
)
# ── MITM CONNECT (apps_script mode) ───────────────────────────
async def _do_mitm_connect(self, host: str, port: int, reader, writer):
"""Intercept TLS, decrypt HTTP, and relay through Apps Script."""
ssl_ctx = self.mitm.get_server_context(host)
# Upgrade the existing connection to TLS (we are the server)
loop = asyncio.get_event_loop()
transport = writer.transport
protocol = transport.get_protocol()
try:
new_transport = await loop.start_tls(
transport, protocol, ssl_ctx, server_side=True,
)
except Exception as e:
log.error("TLS handshake failed for %s: %s", host, e)
return
# Update writer to use the new TLS transport
writer._transport = new_transport
# Read and relay HTTP requests from the browser (now decrypted)
while True:
try:
first_line = await asyncio.wait_for(reader.readline(), timeout=120)
if not first_line:
break
header_block = first_line
while True:
line = await asyncio.wait_for(reader.readline(), timeout=10)
header_block += line
if line in (b"\r\n", b"\n", b""):
break
# Read body
body = b""
for raw_line in header_block.split(b"\r\n"):
if raw_line.lower().startswith(b"content-length:"):
length = int(raw_line.split(b":", 1)[1].strip())
body = await reader.readexactly(length)
break
# Parse the request
request_line = first_line.decode(errors="replace").strip()
parts = request_line.split(" ", 2)
if len(parts) < 2:
break
method = parts[0]
path = parts[1]
# Parse headers
headers = {}
for raw_line in header_block.split(b"\r\n")[1:]:
if b":" in raw_line:
k, v = raw_line.decode(errors="replace").split(":", 1)
headers[k.strip()] = v.strip()
# Build full URL (browser sends just the path in CONNECT)
if port == 443:
url = f"https://{host}{path}"
else:
url = f"https://{host}:{port}{path}"
log.info("MITM → %s %s", method, url)
# Check local cache first (GET only)
response = None
if method == "GET" and not body:
response = self._cache.get(url)
if response:
log.debug("Cache HIT: %s", url[:60])
if response is None:
# Relay through Apps Script
try:
response = await self._relay_smart(method, url, headers, body)
except Exception as e:
log.error("Relay error (%s): %s", url[:60], e)
err_body = f"Relay error: {e}".encode()
response = (
b"HTTP/1.1 502 Bad Gateway\r\n"
b"Content-Type: text/plain\r\n"
b"Content-Length: " + str(len(err_body)).encode() + b"\r\n"
b"\r\n" + err_body
)
# Cache successful GET responses
if method == "GET" and not body and response:
ttl = ResponseCache.parse_ttl(response, url)
if ttl > 0:
self._cache.put(url, response, ttl)
log.debug("Cached (%ds): %s", ttl, url[:60])
writer.write(response)
await writer.drain()
except asyncio.TimeoutError:
break
except asyncio.IncompleteReadError:
break
except ConnectionError:
break
except Exception as e:
log.error("MITM handler error (%s): %s", host, e)
break
async def _relay_smart(self, method, url, headers, body):
"""Choose optimal relay strategy based on request type.
ALL GET requests go through relay_parallel: it does one probe
request and only splits into parallel chunks if the response
is large and the server supports ranges. Small responses still
use a single request (no overhead).
"""
if method == "GET" and not body:
# Skip parallel-range if the client already sent a Range header
# (we must forward it verbatim, not modify it).
if headers:
for k in headers:
if k.lower() == "range":
return await self.fronter.relay(
method, url, headers, body
)
return await self.fronter.relay_parallel(
method, url, headers, body
)
return await self.fronter.relay(method, url, headers, body)
def _is_likely_download(self, url: str, headers: dict) -> bool:
"""Heuristic: is this URL likely a large file download?"""
# Check file extension
path = url.split("?")[0].lower()
large_exts = {
".zip", ".tar", ".gz", ".bz2", ".xz", ".7z", ".rar",
".exe", ".msi", ".dmg", ".deb", ".rpm", ".apk",
".iso", ".img",
".mp4", ".mkv", ".avi", ".mov", ".webm",
".mp3", ".flac", ".wav", ".aac",
".pdf", ".doc", ".docx", ".ppt", ".pptx",
".wasm",
}
for ext in large_exts:
if path.endswith(ext):
return True
return False
# ── Plain HTTP forwarding ─────────────────────────────────────
async def _do_http(self, header_block: bytes, reader, writer):
body = b""
for raw_line in header_block.split(b"\r\n"):
if raw_line.lower().startswith(b"content-length:"):
length = int(raw_line.split(b":", 1)[1].strip())
body = await reader.readexactly(length)
break
first_line = header_block.split(b"\r\n")[0].decode(errors="replace")
log.info("HTTP → %s", first_line)
if self.mode == "apps_script":
# Parse request and relay through Apps Script
parts = first_line.strip().split(" ", 2)
method = parts[0] if parts else "GET"
url = parts[1] if len(parts) > 1 else "/"
headers = {}
for raw_line in header_block.split(b"\r\n")[1:]:
if b":" in raw_line:
k, v = raw_line.decode(errors="replace").split(":", 1)
headers[k.strip()] = v.strip()
# Cache check for GET
response = None
if method == "GET" and not body:
response = self._cache.get(url)
if response:
log.debug("Cache HIT (HTTP): %s", url[:60])
if response is None:
response = await self._relay_smart(method, url, headers, body)
# Cache successful GET
if method == "GET" and not body and response:
ttl = ResponseCache.parse_ttl(response, url)
if ttl > 0:
self._cache.put(url, response, ttl)
elif self.mode in ("google_fronting", "custom_domain", "domain_fronting"):
# Use WebSocket tunnel for ALL traffic (much faster than forward())
response = await self._tunnel_http(header_block, body)
else:
response = await self.fronter.forward(header_block + body)
writer.write(response)
await writer.drain()
async def _tunnel_http(self, header_block: bytes, body: bytes) -> bytes:
"""Forward plain HTTP via a persistent WebSocket tunnel.
Instead of opening a new TLS+HTTP connection for each request
(the old forward() path), this keeps a WebSocket tunnel open
to the target host and pipes raw HTTP through it.
Much faster for rapid-fire requests (e.g., Telegram API).
"""
import re as _re
# Parse target host:port from the raw HTTP request
host = ""
port = 80
for line in header_block.split(b"\r\n")[1:]:
if not line:
break
if line.lower().startswith(b"host:"):
host_val = line.split(b":", 1)[1].strip().decode(errors="replace")
if ":" in host_val:
h, p = host_val.rsplit(":", 1)
try:
host, port = h, int(p)
except ValueError:
host = host_val
else:
host = host_val
break
if not host:
return b"HTTP/1.1 400 Bad Request\r\n\r\nNo Host header\r\n"
# Rewrite the request line: browser sends absolute URL
# (e.g., "GET http://host/path HTTP/1.1") but the target
# server expects a relative path ("GET /path HTTP/1.1")
first_line = header_block.split(b"\r\n")[0]
first_str = first_line.decode(errors="replace")
parts = first_str.split(" ", 2)
if len(parts) >= 2 and parts[1].startswith("http://"):
from urllib.parse import urlparse
parsed = urlparse(parts[1])
rel_path = parsed.path or "/"
if parsed.query:
rel_path += "?" + parsed.query
new_first = f"{parts[0]} {rel_path}"
if len(parts) == 3:
new_first += f" {parts[2]}"
header_block = new_first.encode() + b"\r\n" + b"\r\n".join(header_block.split(b"\r\n")[1:])
raw_request = header_block + body
# Send through tunnel
try:
return await asyncio.wait_for(
self.fronter.forward(raw_request), timeout=30
)
except Exception as e:
log.error("Tunnel HTTP failed (%s:%d): %s", host, port, e)
return b"HTTP/1.1 502 Bad Gateway\r\n\r\nTunnel forward failed\r\n"
+8
View File
@@ -0,0 +1,8 @@
# Core (no external dependencies for basic modes)
# Python 3.10+ required
# Optional: MITM interception (apps_script mode)
cryptography>=41.0.0
# Optional: HTTP/2 multiplexing (faster apps_script relay)
h2>=4.1.0
+76
View File
@@ -0,0 +1,76 @@
"""
Minimal WebSocket frame encoder / decoder (RFC 6455).
Only handles binary (opcode 0x02) and close (opcode 0x08) frames.
Client-to-server frames are always masked as required by the spec.
"""
import os
import struct
def ws_encode(data: bytes, opcode: int = 0x02) -> bytes:
"""Encode *data* into a masked binary WebSocket frame."""
head = bytearray([0x80 | opcode]) # FIN + opcode
length = len(data)
if length < 126:
head.append(0x80 | length)
elif length < 0x10000:
head.append(0x80 | 126)
head += struct.pack("!H", length)
else:
head.append(0x80 | 127)
head += struct.pack("!Q", length)
mask = os.urandom(4)
head += mask
masked = bytearray(data)
for i in range(len(masked)):
masked[i] ^= mask[i & 3]
return bytes(head) + bytes(masked)
def ws_decode(buf: bytes):
"""Try to decode one frame from *buf*.
Returns ``(opcode, payload, consumed_bytes)`` or ``None`` if the
buffer does not yet contain a complete frame.
"""
if len(buf) < 2:
return None
opcode = buf[0] & 0x0F
is_masked = buf[1] & 0x80
length = buf[1] & 0x7F
pos = 2
if length == 126:
if len(buf) < 4:
return None
length = struct.unpack("!H", buf[2:4])[0]
pos = 4
elif length == 127:
if len(buf) < 10:
return None
length = struct.unpack("!Q", buf[2:10])[0]
pos = 10
mask = None
if is_masked:
if len(buf) < pos + 4:
return None
mask = buf[pos : pos + 4]
pos += 4
if len(buf) < pos + length:
return None
payload = bytearray(buf[pos : pos + length])
if mask:
for i in range(len(payload)):
payload[i] ^= mask[i & 3]
return opcode, bytes(payload), pos + length