mirror of
https://github.com/masterking32/MasterHttpRelayVPN.git
synced 2026-05-17 21:24:37 +03:00
First commit! (TESTING)
This commit is contained in:
+35
@@ -0,0 +1,35 @@
|
||||
# Secrets & user config
|
||||
config.json
|
||||
.env
|
||||
|
||||
# CA certificates (generated at runtime, contain private keys)
|
||||
ca/
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
*.egg
|
||||
|
||||
# Virtual environments
|
||||
venv/
|
||||
.venv/
|
||||
env/
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Temp MITM certs
|
||||
domainfront_certs_*/
|
||||
@@ -0,0 +1,222 @@
|
||||
# DomainFront Tunnel (MasterHttpRelayVPN Testing Python!)
|
||||
|
||||
A local HTTP proxy that bypasses DPI (Deep Packet Inspection) censorship using **domain fronting**. The proxy tunnels all browser traffic through a CDN — the TLS SNI shows an allowed domain (e.g. `www.google.com`) while the encrypted HTTP Host header routes to your relay endpoint.
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
Browser ──► Local Proxy ──► CDN (TLS SNI: www.google.com) ──► Your Relay ──► Target Website
|
||||
DPI sees: "www.google.com" ✓
|
||||
Actual destination: hidden
|
||||
```
|
||||
|
||||
The DPI/firewall only sees the SNI field in the TLS handshake, which shows an innocuous, unblockable domain. The real destination is hidden inside the encrypted HTTP stream.
|
||||
|
||||
## Supported Modes
|
||||
|
||||
| Mode | Relay | Description |
|
||||
|------|-------|-------------|
|
||||
| `apps_script` | Google Apps Script | Fronts through `www.google.com` → `script.google.com`. Free, no server needed. |
|
||||
| `google_fronting` | Google Cloud Run | Fronts through Google IP → your Cloud Run service. |
|
||||
| `domain_fronting` | Cloudflare Worker | Classic domain fronting via Cloudflare CDN. |
|
||||
| `custom_domain` | Custom domain on CF | Direct connection to your custom domain on Cloudflare. |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Install
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/masterking32/MasterHttpRelayVPN.git
|
||||
cd domainfront-tunnel
|
||||
|
||||
# (Optional) Create a virtual environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate # Linux/macOS
|
||||
venv\Scripts\activate # Windows
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
> **Note:** Python 3.10+ is required. Core functionality has no external dependencies. The optional packages (`cryptography`, `h2`) enable MITM interception and HTTP/2 multiplexing for the `apps_script` mode.
|
||||
|
||||
### 2. Configure
|
||||
|
||||
```bash
|
||||
cp config.example.json config.json
|
||||
```
|
||||
|
||||
Edit `config.json` with your values:
|
||||
|
||||
```json
|
||||
{
|
||||
"mode": "apps_script",
|
||||
"google_ip": "216.239.38.120",
|
||||
"front_domain": "www.google.com",
|
||||
"script_id": "YOUR_APPS_SCRIPT_DEPLOYMENT_ID",
|
||||
"auth_key": "your-strong-secret-key",
|
||||
"listen_host": "127.0.0.1",
|
||||
"listen_port": 8085,
|
||||
"log_level": "INFO",
|
||||
"verify_ssl": true
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Run
|
||||
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
### 4. Configure Your Browser
|
||||
|
||||
Set your browser's HTTP proxy to `127.0.0.1:8085` (or whatever `listen_host`:`listen_port` you configured).
|
||||
|
||||
For `apps_script` mode, you also need to install the generated CA certificate (`ca/ca.crt`) in your browser's trusted root CAs.
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Required Fields
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `mode` | One of: `apps_script`, `google_fronting`, `domain_fronting`, `custom_domain` |
|
||||
| `auth_key` | Shared secret between the proxy and your relay endpoint |
|
||||
|
||||
### Mode-Specific Fields
|
||||
|
||||
| Field | Modes | Description |
|
||||
|-------|-------|-------------|
|
||||
| `script_id` | `apps_script` | Your deployed Apps Script ID (or array of IDs for load balancing) |
|
||||
| `worker_host` | `domain_fronting`, `google_fronting` | Your Worker/Cloud Run hostname |
|
||||
| `custom_domain` | `custom_domain` | Your custom domain on Cloudflare |
|
||||
| `front_domain` | `domain_fronting`, `google_fronting`, `apps_script` | The domain shown in TLS SNI (default: `www.google.com`) |
|
||||
| `google_ip` | `google_fronting`, `apps_script` | Google IP to connect to (default: `216.239.38.120`) |
|
||||
|
||||
### Optional Fields
|
||||
|
||||
| Field | Default | Description |
|
||||
|-------|---------|-------------|
|
||||
| `listen_host` | `127.0.0.1` | Local proxy bind address |
|
||||
| `listen_port` | `8080` | Local proxy port |
|
||||
| `log_level` | `INFO` | Logging level: `DEBUG`, `INFO`, `WARNING`, `ERROR` |
|
||||
| `verify_ssl` | `true` | Verify TLS certificates |
|
||||
| `worker_path` | `""` | URL path prefix for the worker |
|
||||
| `script_ids` | — | Array of Apps Script IDs for round-robin load balancing |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All settings can be overridden via environment variables (useful for containers/CI):
|
||||
|
||||
| Variable | Overrides |
|
||||
|----------|-----------|
|
||||
| `DFT_CONFIG` | Config file path |
|
||||
| `DFT_AUTH_KEY` | `auth_key` |
|
||||
| `DFT_SCRIPT_ID` | `script_id` |
|
||||
| `DFT_PORT` | `listen_port` |
|
||||
| `DFT_HOST` | `listen_host` |
|
||||
| `DFT_LOG_LEVEL` | `log_level` |
|
||||
|
||||
## CLI Usage
|
||||
|
||||
```
|
||||
usage: domainfront-tunnel [-h] [-c CONFIG] [-p PORT] [--host HOST]
|
||||
[--log-level {DEBUG,INFO,WARNING,ERROR}] [-v]
|
||||
|
||||
options:
|
||||
-c, --config CONFIG Path to config file (default: config.json)
|
||||
-p, --port PORT Override listen port
|
||||
--host HOST Override listen host
|
||||
--log-level LEVEL Override log level
|
||||
-v, --version Show version and exit
|
||||
```
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Basic usage
|
||||
python main.py
|
||||
|
||||
# Custom config file
|
||||
python main.py -c /path/to/my-config.json
|
||||
|
||||
# Override port
|
||||
python main.py -p 9090
|
||||
|
||||
# Debug logging
|
||||
python main.py --log-level DEBUG
|
||||
|
||||
# Using environment variables
|
||||
DFT_AUTH_KEY=my-secret DFT_PORT=9090 python main.py
|
||||
```
|
||||
|
||||
## Apps Script Setup
|
||||
|
||||
1. Go to [Google Apps Script](https://script.google.com/) and create a new project.
|
||||
2. Paste your relay script code into `Code.gs`.
|
||||
3. Deploy as a **Web App**:
|
||||
- Execute as: **Me**
|
||||
- Who has access: **Anyone**
|
||||
4. Copy the **Deployment ID** and paste it into `config.json` as `script_id`.
|
||||
5. Set a strong `auth_key` in both the Apps Script and `config.json`.
|
||||
|
||||
### Multiple Script IDs (Load Balancing)
|
||||
|
||||
For higher throughput, deploy multiple copies and use an array:
|
||||
|
||||
```json
|
||||
{
|
||||
"script_ids": [
|
||||
"DEPLOYMENT_ID_1",
|
||||
"DEPLOYMENT_ID_2",
|
||||
"DEPLOYMENT_ID_3"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────┐ ┌──────────────┐ ┌─────────────┐ ┌──────────┐
|
||||
│ Browser │────►│ Local Proxy │────►│ CDN / Google │────►│ Relay │──► Internet
|
||||
│ │◄────│ (this tool) │◄────│ (fronted) │◄────│ Endpoint │◄──
|
||||
└─────────┘ └──────────────┘ └─────────────┘ └──────────┘
|
||||
HTTP/CONNECT TLS (SNI: ok) Fetch target
|
||||
MITM (optional) Host: relay Return response
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `main.py` | Entry point, config loading, CLI |
|
||||
| `proxy_server.py` | Local HTTP/CONNECT proxy server |
|
||||
| `domain_fronter.py` | Domain fronting engine, connection pooling, relay logic |
|
||||
| `h2_transport.py` | HTTP/2 multiplexed transport (optional, for performance) |
|
||||
| `mitm.py` | MITM certificate manager for HTTPS interception |
|
||||
| `ws.py` | WebSocket frame encoder/decoder (RFC 6455) |
|
||||
|
||||
## Performance Features
|
||||
|
||||
- **HTTP/2 multiplexing**: Single TLS connection handles 100+ concurrent requests
|
||||
- **Connection pooling**: Pre-warmed TLS connection pool with automatic maintenance
|
||||
- **Request batching**: Groups concurrent requests into single relay calls
|
||||
- **Request coalescing**: Deduplicates identical concurrent GET requests
|
||||
- **Parallel range downloads**: Splits large downloads into concurrent chunks
|
||||
- **Response caching**: LRU cache for static assets (configurable, 50 MB default)
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **Never commit `config.json`** — it contains your `auth_key`. The `.gitignore` excludes it.
|
||||
- The `ca/` directory contains your generated CA private key. Keep it secure.
|
||||
- Use a strong, unique `auth_key` to prevent unauthorized use of your relay.
|
||||
- Set `listen_host` to `127.0.0.1` (not `0.0.0.0`) unless you need LAN access.
|
||||
|
||||
## Special Thanks
|
||||
|
||||
Special thanks to [@abolix](https://github.com/abolix) for making this project possible.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"_comment": "Copy this file to config.json and fill in your values",
|
||||
"mode": "apps_script",
|
||||
"google_ip": "216.239.38.120",
|
||||
"front_domain": "www.google.com",
|
||||
"script_id": "YOUR_APPS_SCRIPT_DEPLOYMENT_ID",
|
||||
"auth_key": "CHANGE_ME_TO_A_STRONG_SECRET",
|
||||
"listen_host": "127.0.0.1",
|
||||
"listen_port": 8085,
|
||||
"log_level": "INFO",
|
||||
"verify_ssl": true
|
||||
}
|
||||
+1159
File diff suppressed because it is too large
Load Diff
+419
@@ -0,0 +1,419 @@
|
||||
"""
|
||||
HTTP/2 multiplexed transport for domain-fronted connections.
|
||||
|
||||
One TLS connection → many concurrent HTTP/2 streams → massive throughput.
|
||||
Eliminates per-request TLS handshake overhead entirely.
|
||||
|
||||
Instead of a pool of 30 HTTP/1.1 connections (each handling 1 request),
|
||||
this uses a SINGLE HTTP/2 connection handling 100+ concurrent requests.
|
||||
|
||||
Performance comparison:
|
||||
HTTP/1.1 pool: 30 connections × 1 request = 30 concurrent requests max
|
||||
HTTP/2 mux: 1 connection × 100 streams = 100 concurrent requests
|
||||
|
||||
Requires: pip install h2
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import gzip
|
||||
import logging
|
||||
import socket
|
||||
import ssl
|
||||
from urllib.parse import urlparse
|
||||
|
||||
log = logging.getLogger("H2")
|
||||
|
||||
try:
|
||||
import h2.connection
|
||||
import h2.config
|
||||
import h2.events
|
||||
import h2.settings
|
||||
H2_AVAILABLE = True
|
||||
except ImportError:
|
||||
H2_AVAILABLE = False
|
||||
|
||||
|
||||
class _StreamState:
|
||||
"""State for a single in-flight HTTP/2 stream."""
|
||||
__slots__ = ("status", "headers", "data", "done", "error")
|
||||
|
||||
def __init__(self):
|
||||
self.status = 0
|
||||
self.headers: dict[str, str] = {}
|
||||
self.data = bytearray()
|
||||
self.done = asyncio.Event()
|
||||
self.error: str | None = None
|
||||
|
||||
|
||||
class H2Transport:
|
||||
"""
|
||||
Persistent HTTP/2 connection with automatic stream multiplexing.
|
||||
|
||||
All relay requests share ONE TLS connection. Each request becomes
|
||||
an independent HTTP/2 stream, running fully concurrently.
|
||||
|
||||
Features:
|
||||
- Auto-connect on first use
|
||||
- Auto-reconnect on connection loss
|
||||
- Redirect following (as new streams, same connection)
|
||||
- Gzip decompression
|
||||
- Configurable max concurrency
|
||||
"""
|
||||
|
||||
def __init__(self, connect_host: str, sni_host: str,
|
||||
verify_ssl: bool = True):
|
||||
self.connect_host = connect_host
|
||||
self.sni_host = sni_host
|
||||
self.verify_ssl = verify_ssl
|
||||
|
||||
self._reader: asyncio.StreamReader | None = None
|
||||
self._writer: asyncio.StreamWriter | None = None
|
||||
self._h2: "h2.connection.H2Connection | None" = None
|
||||
self._connected = False
|
||||
|
||||
self._write_lock = asyncio.Lock()
|
||||
self._connect_lock = asyncio.Lock()
|
||||
self._read_task: asyncio.Task | None = None
|
||||
|
||||
# Per-stream tracking
|
||||
self._streams: dict[int, _StreamState] = {}
|
||||
|
||||
# Stats
|
||||
self.total_requests = 0
|
||||
self.total_streams = 0
|
||||
|
||||
# ── Connection lifecycle ──────────────────────────────────────
|
||||
|
||||
@property
|
||||
def is_connected(self) -> bool:
|
||||
return self._connected
|
||||
|
||||
async def ensure_connected(self):
|
||||
"""Connect if not already connected."""
|
||||
if self._connected:
|
||||
return
|
||||
async with self._connect_lock:
|
||||
if self._connected:
|
||||
return
|
||||
await self._do_connect()
|
||||
|
||||
async def _do_connect(self):
|
||||
"""Establish the HTTP/2 connection with optimized socket settings."""
|
||||
ctx = ssl.create_default_context()
|
||||
# Advertise both h2 and http/1.1 — some DPI blocks h2-only ALPN
|
||||
ctx.set_alpn_protocols(["h2", "http/1.1"])
|
||||
if not self.verify_ssl:
|
||||
ctx.check_hostname = False
|
||||
ctx.verify_mode = ssl.CERT_NONE
|
||||
|
||||
# Create raw TCP socket with TCP_NODELAY BEFORE TLS handshake.
|
||||
# Nagle's algorithm can delay small writes (H2 frames) by up to 200ms
|
||||
# waiting to coalesce — TCP_NODELAY forces immediate send.
|
||||
raw = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
raw.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
|
||||
raw.setblocking(False)
|
||||
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
asyncio.get_event_loop().sock_connect(
|
||||
raw, (self.connect_host, 443)
|
||||
),
|
||||
timeout=15,
|
||||
)
|
||||
self._reader, self._writer = await asyncio.wait_for(
|
||||
asyncio.open_connection(
|
||||
ssl=ctx,
|
||||
server_hostname=self.sni_host,
|
||||
sock=raw,
|
||||
),
|
||||
timeout=15,
|
||||
)
|
||||
except Exception:
|
||||
raw.close()
|
||||
raise
|
||||
|
||||
# Verify we actually got HTTP/2
|
||||
ssl_obj = self._writer.get_extra_info("ssl_object")
|
||||
negotiated = ssl_obj.selected_alpn_protocol() if ssl_obj else None
|
||||
if negotiated != "h2":
|
||||
self._writer.close()
|
||||
raise RuntimeError(
|
||||
f"H2 ALPN negotiation failed (got {negotiated!r})"
|
||||
)
|
||||
|
||||
config = h2.config.H2Configuration(
|
||||
client_side=True,
|
||||
header_encoding="utf-8",
|
||||
)
|
||||
self._h2 = h2.connection.H2Connection(config=config)
|
||||
self._h2.initiate_connection()
|
||||
|
||||
# Connection-level flow control: ~16MB window
|
||||
self._h2.increment_flow_control_window(2 ** 24 - 65535)
|
||||
|
||||
# Per-stream settings: 1MB initial window, disable server push
|
||||
self._h2.update_settings({
|
||||
h2.settings.SettingCodes.INITIAL_WINDOW_SIZE: 1 * 1024 * 1024,
|
||||
h2.settings.SettingCodes.ENABLE_PUSH: 0,
|
||||
})
|
||||
|
||||
await self._flush()
|
||||
|
||||
self._connected = True
|
||||
self._read_task = asyncio.create_task(self._reader_loop())
|
||||
log.info("H2 connected → %s (SNI=%s, TCP_NODELAY=on)",
|
||||
self.connect_host, self.sni_host)
|
||||
|
||||
async def reconnect(self):
|
||||
"""Close current connection and re-establish."""
|
||||
await self._close_internal()
|
||||
await self._do_connect()
|
||||
|
||||
async def _close_internal(self):
|
||||
self._connected = False
|
||||
if self._read_task:
|
||||
self._read_task.cancel()
|
||||
self._read_task = None
|
||||
if self._writer:
|
||||
try:
|
||||
self._writer.close()
|
||||
except Exception:
|
||||
pass
|
||||
self._writer = None
|
||||
# Wake all pending streams so they can raise
|
||||
for state in self._streams.values():
|
||||
state.error = "Connection closed"
|
||||
state.done.set()
|
||||
self._streams.clear()
|
||||
|
||||
# ── Public API ────────────────────────────────────────────────
|
||||
|
||||
async def request(self, method: str, path: str, host: str,
|
||||
headers: dict | None = None,
|
||||
body: bytes | None = None,
|
||||
timeout: float = 25,
|
||||
follow_redirects: int = 5) -> tuple[int, dict, bytes]:
|
||||
"""
|
||||
Send an HTTP/2 request and return (status, headers, body).
|
||||
|
||||
Thread-safe: many concurrent calls each get their own stream.
|
||||
Redirects are followed as new streams on the same connection.
|
||||
"""
|
||||
await self.ensure_connected()
|
||||
self.total_requests += 1
|
||||
|
||||
for _ in range(follow_redirects + 1):
|
||||
status, resp_headers, resp_body = await self._single_request(
|
||||
method, path, host, headers, body, timeout,
|
||||
)
|
||||
|
||||
if status not in (301, 302, 303, 307, 308):
|
||||
return status, resp_headers, resp_body
|
||||
|
||||
location = resp_headers.get("location", "")
|
||||
if not location:
|
||||
return status, resp_headers, resp_body
|
||||
|
||||
parsed = urlparse(location)
|
||||
path = parsed.path + ("?" + parsed.query if parsed.query else "")
|
||||
host = parsed.netloc or host
|
||||
method = "GET"
|
||||
body = None
|
||||
headers = None # Drop request headers on redirect
|
||||
|
||||
return status, resp_headers, resp_body
|
||||
|
||||
# ── Stream handling ───────────────────────────────────────────
|
||||
|
||||
async def _single_request(self, method, path, host, headers, body,
|
||||
timeout) -> tuple[int, dict, bytes]:
|
||||
"""Send one HTTP/2 request on a new stream, wait for response."""
|
||||
if not self._connected:
|
||||
await self.ensure_connected()
|
||||
|
||||
stream_id = None
|
||||
|
||||
async with self._write_lock:
|
||||
try:
|
||||
stream_id = self._h2.get_next_available_stream_id()
|
||||
except Exception:
|
||||
# Connection is stale — reconnect
|
||||
await self.reconnect()
|
||||
stream_id = self._h2.get_next_available_stream_id()
|
||||
|
||||
h2_headers = [
|
||||
(":method", method),
|
||||
(":path", path),
|
||||
(":authority", host),
|
||||
(":scheme", "https"),
|
||||
("accept-encoding", "gzip"),
|
||||
]
|
||||
if headers:
|
||||
for k, v in headers.items():
|
||||
h2_headers.append((k.lower(), str(v)))
|
||||
|
||||
end_stream = not body
|
||||
self._h2.send_headers(stream_id, h2_headers, end_stream=end_stream)
|
||||
|
||||
if body:
|
||||
# Send body (may need chunking for flow control)
|
||||
self._send_body(stream_id, body)
|
||||
|
||||
state = _StreamState()
|
||||
self._streams[stream_id] = state
|
||||
self.total_streams += 1
|
||||
|
||||
await self._flush()
|
||||
|
||||
# Wait for complete response
|
||||
try:
|
||||
await asyncio.wait_for(state.done.wait(), timeout=timeout)
|
||||
except asyncio.TimeoutError:
|
||||
self._streams.pop(stream_id, None)
|
||||
raise TimeoutError(
|
||||
f"H2 stream {stream_id} timed out ({timeout}s)"
|
||||
)
|
||||
|
||||
self._streams.pop(stream_id, None)
|
||||
|
||||
if state.error:
|
||||
raise ConnectionError(f"H2 stream error: {state.error}")
|
||||
|
||||
# Auto-decompress gzip
|
||||
resp_body = bytes(state.data)
|
||||
if state.headers.get("content-encoding", "").lower() == "gzip":
|
||||
try:
|
||||
resp_body = gzip.decompress(resp_body)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return state.status, state.headers, resp_body
|
||||
|
||||
def _send_body(self, stream_id: int, body: bytes):
|
||||
"""Send request body, respecting H2 flow control window."""
|
||||
# For small bodies (typical JSON payloads), send in one shot
|
||||
while body:
|
||||
max_size = self._h2.local_settings.max_frame_size
|
||||
window = self._h2.local_flow_control_window(stream_id)
|
||||
send_size = min(len(body), max_size, window)
|
||||
if send_size <= 0:
|
||||
# Flow control full — let the reader loop process
|
||||
# window updates before we continue
|
||||
break
|
||||
end = send_size >= len(body)
|
||||
self._h2.send_data(stream_id, body[:send_size], end_stream=end)
|
||||
body = body[send_size:]
|
||||
|
||||
# ── Background reader ─────────────────────────────────────────
|
||||
|
||||
async def _reader_loop(self):
|
||||
"""Background: read H2 frames, dispatch events to waiting streams."""
|
||||
try:
|
||||
while self._connected:
|
||||
data = await self._reader.read(65536)
|
||||
if not data:
|
||||
log.warning("H2 remote closed connection")
|
||||
break
|
||||
|
||||
try:
|
||||
events = self._h2.receive_data(data)
|
||||
except Exception as e:
|
||||
log.error("H2 protocol error: %s", e)
|
||||
break
|
||||
|
||||
for event in events:
|
||||
self._dispatch(event)
|
||||
|
||||
# Send pending data (acks, window updates, ping responses)
|
||||
async with self._write_lock:
|
||||
await self._flush()
|
||||
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
except Exception as e:
|
||||
log.error("H2 reader error: %s", e)
|
||||
finally:
|
||||
self._connected = False
|
||||
for state in self._streams.values():
|
||||
if not state.done.is_set():
|
||||
state.error = "Connection lost"
|
||||
state.done.set()
|
||||
log.info("H2 reader loop ended")
|
||||
|
||||
def _dispatch(self, event):
|
||||
"""Route a single h2 event to its stream."""
|
||||
if isinstance(event, h2.events.ResponseReceived):
|
||||
state = self._streams.get(event.stream_id)
|
||||
if state:
|
||||
for name, value in event.headers:
|
||||
n = name if isinstance(name, str) else name.decode()
|
||||
v = value if isinstance(value, str) else value.decode()
|
||||
if n == ":status":
|
||||
state.status = int(v)
|
||||
else:
|
||||
state.headers[n] = v
|
||||
|
||||
elif isinstance(event, h2.events.DataReceived):
|
||||
state = self._streams.get(event.stream_id)
|
||||
if state:
|
||||
state.data.extend(event.data)
|
||||
# Always acknowledge received data for flow control
|
||||
self._h2.acknowledge_received_data(
|
||||
event.flow_controlled_length, event.stream_id
|
||||
)
|
||||
|
||||
elif isinstance(event, h2.events.StreamEnded):
|
||||
state = self._streams.get(event.stream_id)
|
||||
if state:
|
||||
state.done.set()
|
||||
|
||||
elif isinstance(event, h2.events.StreamReset):
|
||||
state = self._streams.get(event.stream_id)
|
||||
if state:
|
||||
state.error = f"Stream reset (code={event.error_code})"
|
||||
state.done.set()
|
||||
|
||||
elif isinstance(event, h2.events.WindowUpdated):
|
||||
pass # h2 library handles window bookkeeping
|
||||
|
||||
elif isinstance(event, h2.events.SettingsAcknowledged):
|
||||
pass
|
||||
|
||||
elif isinstance(event, h2.events.PingReceived):
|
||||
pass # h2 library auto-responds
|
||||
|
||||
elif isinstance(event, h2.events.PingAckReceived):
|
||||
pass # keepalive confirmed
|
||||
|
||||
# ── Internal ──────────────────────────────────────────────────
|
||||
|
||||
async def _flush(self):
|
||||
"""Write pending H2 frame data to the socket."""
|
||||
data = self._h2.data_to_send()
|
||||
if data and self._writer:
|
||||
self._writer.write(data)
|
||||
await self._writer.drain()
|
||||
|
||||
async def close(self):
|
||||
"""Gracefully close the HTTP/2 connection."""
|
||||
if self._h2 and self._connected:
|
||||
try:
|
||||
self._h2.close_connection()
|
||||
async with self._write_lock:
|
||||
await self._flush()
|
||||
except Exception:
|
||||
pass
|
||||
await self._close_internal()
|
||||
|
||||
async def ping(self):
|
||||
"""Send an H2 PING frame to keep the connection alive."""
|
||||
if not self._connected or not self._h2:
|
||||
return
|
||||
try:
|
||||
async with self._write_lock:
|
||||
if not self._connected:
|
||||
return
|
||||
self._h2.ping(b"\x00" * 8)
|
||||
await self._flush()
|
||||
except Exception as e:
|
||||
log.debug("H2 PING failed: %s", e)
|
||||
@@ -0,0 +1,164 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
DomainFront Tunnel — Bypass DPI censorship via Domain Fronting.
|
||||
|
||||
Run a local HTTP proxy that tunnels all traffic through a CDN using
|
||||
domain fronting: the TLS SNI shows an allowed domain while the encrypted
|
||||
HTTP Host header routes to your Cloudflare Worker relay.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
from proxy_server import ProxyServer
|
||||
|
||||
__version__ = "1.0.0"
|
||||
|
||||
|
||||
def setup_logging(level_name: str):
|
||||
level = getattr(logging, level_name.upper(), logging.INFO)
|
||||
logging.basicConfig(
|
||||
level=level,
|
||||
format="%(asctime)s [%(name)-12s] %(levelname)-7s %(message)s",
|
||||
datefmt="%H:%M:%S",
|
||||
)
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="domainfront-tunnel",
|
||||
description="Local HTTP proxy that tunnels traffic through domain fronting.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-c", "--config",
|
||||
default=os.environ.get("DFT_CONFIG", "config.json"),
|
||||
help="Path to config file (default: config.json, env: DFT_CONFIG)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-p", "--port",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Override listen port (env: DFT_PORT)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--host",
|
||||
default=None,
|
||||
help="Override listen host (env: DFT_HOST)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--log-level",
|
||||
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
|
||||
default=None,
|
||||
help="Override log level (env: DFT_LOG_LEVEL)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-v", "--version",
|
||||
action="version",
|
||||
version=f"%(prog)s {__version__}",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
config_path = args.config
|
||||
|
||||
try:
|
||||
with open(config_path) as f:
|
||||
config = json.load(f)
|
||||
except FileNotFoundError:
|
||||
print(f"Config not found: {config_path}")
|
||||
print("Copy config.example.json to config.json and fill in your values.")
|
||||
sys.exit(1)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"Invalid JSON in config: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# Environment variable overrides
|
||||
if os.environ.get("DFT_AUTH_KEY"):
|
||||
config["auth_key"] = os.environ["DFT_AUTH_KEY"]
|
||||
if os.environ.get("DFT_SCRIPT_ID"):
|
||||
config["script_id"] = os.environ["DFT_SCRIPT_ID"]
|
||||
|
||||
# CLI argument overrides
|
||||
if args.port is not None:
|
||||
config["listen_port"] = args.port
|
||||
elif os.environ.get("DFT_PORT"):
|
||||
config["listen_port"] = int(os.environ["DFT_PORT"])
|
||||
|
||||
if args.host is not None:
|
||||
config["listen_host"] = args.host
|
||||
elif os.environ.get("DFT_HOST"):
|
||||
config["listen_host"] = os.environ["DFT_HOST"]
|
||||
|
||||
if args.log_level is not None:
|
||||
config["log_level"] = args.log_level
|
||||
elif os.environ.get("DFT_LOG_LEVEL"):
|
||||
config["log_level"] = os.environ["DFT_LOG_LEVEL"]
|
||||
|
||||
for key in ("auth_key",):
|
||||
if key not in config:
|
||||
print(f"Missing required config key: {key}")
|
||||
sys.exit(1)
|
||||
|
||||
mode = config.get("mode", "domain_fronting")
|
||||
if mode == "custom_domain" and "custom_domain" not in config:
|
||||
print("Mode 'custom_domain' requires 'custom_domain' in config")
|
||||
sys.exit(1)
|
||||
if mode == "domain_fronting":
|
||||
for key in ("front_domain", "worker_host"):
|
||||
if key not in config:
|
||||
print(f"Mode 'domain_fronting' requires '{key}' in config")
|
||||
sys.exit(1)
|
||||
if mode == "google_fronting":
|
||||
if "worker_host" not in config:
|
||||
print("Mode 'google_fronting' requires 'worker_host' in config (your Cloud Run URL)")
|
||||
sys.exit(1)
|
||||
if mode == "apps_script":
|
||||
sid = config.get("script_ids") or config.get("script_id")
|
||||
if not sid or (isinstance(sid, str) and sid == "YOUR_APPS_SCRIPT_DEPLOYMENT_ID"):
|
||||
print("Mode 'apps_script' requires 'script_id' in config.")
|
||||
print("Deploy the Apps Script from appsscript/Code.gs and paste the Deployment ID.")
|
||||
sys.exit(1)
|
||||
|
||||
setup_logging(config.get("log_level", "INFO"))
|
||||
log = logging.getLogger("Main")
|
||||
|
||||
mode = config.get("mode", "domain_fronting")
|
||||
log.info("DomainFront Tunnel starting (mode: %s)", mode)
|
||||
|
||||
if mode == "custom_domain":
|
||||
log.info("Custom domain : %s", config["custom_domain"])
|
||||
elif mode == "google_fronting":
|
||||
log.info("Google fronting : SNI=%s → Host=%s",
|
||||
config.get("front_domain", "www.google.com"), config["worker_host"])
|
||||
log.info("Google IP : %s", config.get("google_ip", "216.239.38.120"))
|
||||
elif mode == "apps_script":
|
||||
log.info("Apps Script relay : SNI=%s → script.google.com",
|
||||
config.get("front_domain", "www.google.com"))
|
||||
script_ids = config.get("script_ids") or config.get("script_id")
|
||||
if isinstance(script_ids, list):
|
||||
log.info("Script IDs : %d scripts (round-robin)", len(script_ids))
|
||||
for i, sid in enumerate(script_ids):
|
||||
log.info(" [%d] %s", i + 1, sid)
|
||||
else:
|
||||
log.info("Script ID : %s", script_ids)
|
||||
log.info("MITM enabled — install ca/ca.crt in your browser!")
|
||||
else:
|
||||
log.info("Front domain (SNI) : %s", config.get("front_domain", "?"))
|
||||
log.info("Worker host (Host) : %s", config.get("worker_host", "?"))
|
||||
|
||||
log.info("Proxy address : %s:%d", config.get("listen_host", "127.0.0.1"), config.get("listen_port", 8080))
|
||||
|
||||
try:
|
||||
asyncio.run(ProxyServer(config).start())
|
||||
except KeyboardInterrupt:
|
||||
log.info("Stopped")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,153 @@
|
||||
"""
|
||||
MITM certificate manager for HTTPS interception.
|
||||
|
||||
Generates a CA certificate (once, stored as files) and per-domain
|
||||
certificates (on the fly, cached in memory) so the local proxy can
|
||||
decrypt HTTPS traffic and relay it through Apps Script.
|
||||
|
||||
The user must install ca/ca.crt in their browser's trusted CAs once.
|
||||
|
||||
Requires: pip install cryptography
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import os
|
||||
import ssl
|
||||
import tempfile
|
||||
|
||||
from cryptography import x509
|
||||
from cryptography.hazmat.primitives import hashes, serialization
|
||||
from cryptography.hazmat.primitives.asymmetric import rsa
|
||||
from cryptography.x509.oid import NameOID
|
||||
|
||||
log = logging.getLogger("MITM")
|
||||
|
||||
CA_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "ca")
|
||||
CA_KEY_FILE = os.path.join(CA_DIR, "ca.key")
|
||||
CA_CERT_FILE = os.path.join(CA_DIR, "ca.crt")
|
||||
|
||||
|
||||
class MITMCertManager:
|
||||
def __init__(self):
|
||||
self._ca_key = None
|
||||
self._ca_cert = None
|
||||
self._ctx_cache: dict[str, ssl.SSLContext] = {}
|
||||
self._cert_dir = tempfile.mkdtemp(prefix="domainfront_certs_")
|
||||
self._ensure_ca()
|
||||
|
||||
def _ensure_ca(self):
|
||||
if os.path.exists(CA_KEY_FILE) and os.path.exists(CA_CERT_FILE):
|
||||
with open(CA_KEY_FILE, "rb") as f:
|
||||
self._ca_key = serialization.load_pem_private_key(
|
||||
f.read(), password=None
|
||||
)
|
||||
with open(CA_CERT_FILE, "rb") as f:
|
||||
self._ca_cert = x509.load_pem_x509_certificate(f.read())
|
||||
log.info("Loaded CA from %s", CA_DIR)
|
||||
else:
|
||||
self._create_ca()
|
||||
|
||||
def _create_ca(self):
|
||||
os.makedirs(CA_DIR, exist_ok=True)
|
||||
|
||||
self._ca_key = rsa.generate_private_key(
|
||||
public_exponent=65537, key_size=2048
|
||||
)
|
||||
subject = issuer = x509.Name([
|
||||
x509.NameAttribute(NameOID.COMMON_NAME, "DomainFront Tunnel CA"),
|
||||
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "DomainFront Tunnel"),
|
||||
])
|
||||
now = datetime.datetime.now(datetime.timezone.utc)
|
||||
self._ca_cert = (
|
||||
x509.CertificateBuilder()
|
||||
.subject_name(subject)
|
||||
.issuer_name(issuer)
|
||||
.public_key(self._ca_key.public_key())
|
||||
.serial_number(x509.random_serial_number())
|
||||
.not_valid_before(now)
|
||||
.not_valid_after(now + datetime.timedelta(days=3650))
|
||||
.add_extension(
|
||||
x509.BasicConstraints(ca=True, path_length=0), critical=True
|
||||
)
|
||||
.add_extension(
|
||||
x509.KeyUsage(
|
||||
digital_signature=True,
|
||||
key_cert_sign=True,
|
||||
crl_sign=True,
|
||||
content_commitment=False,
|
||||
key_encipherment=False,
|
||||
data_encipherment=False,
|
||||
key_agreement=False,
|
||||
encipher_only=False,
|
||||
decipher_only=False,
|
||||
),
|
||||
critical=True,
|
||||
)
|
||||
.sign(self._ca_key, hashes.SHA256())
|
||||
)
|
||||
|
||||
with open(CA_KEY_FILE, "wb") as f:
|
||||
f.write(
|
||||
self._ca_key.private_bytes(
|
||||
serialization.Encoding.PEM,
|
||||
serialization.PrivateFormat.TraditionalOpenSSL,
|
||||
serialization.NoEncryption(),
|
||||
)
|
||||
)
|
||||
with open(CA_CERT_FILE, "wb") as f:
|
||||
f.write(self._ca_cert.public_bytes(serialization.Encoding.PEM))
|
||||
|
||||
log.warning("Generated new CA certificate: %s", CA_CERT_FILE)
|
||||
log.warning(">>> Install this file in your browser's Trusted Root CAs! <<<")
|
||||
|
||||
def get_server_context(self, domain: str) -> ssl.SSLContext:
|
||||
if domain not in self._ctx_cache:
|
||||
key_pem, cert_pem = self._generate_domain_cert(domain)
|
||||
|
||||
cert_file = os.path.join(self._cert_dir, f"{domain}.crt")
|
||||
key_file = os.path.join(self._cert_dir, f"{domain}.key")
|
||||
|
||||
ca_pem = self._ca_cert.public_bytes(serialization.Encoding.PEM)
|
||||
with open(cert_file, "wb") as f:
|
||||
f.write(cert_pem + ca_pem)
|
||||
with open(key_file, "wb") as f:
|
||||
f.write(key_pem)
|
||||
|
||||
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
||||
ctx.set_alpn_protocols(["http/1.1"])
|
||||
ctx.load_cert_chain(cert_file, key_file)
|
||||
self._ctx_cache[domain] = ctx
|
||||
|
||||
return self._ctx_cache[domain]
|
||||
|
||||
def _generate_domain_cert(self, domain: str):
|
||||
key = rsa.generate_private_key(
|
||||
public_exponent=65537, key_size=2048
|
||||
)
|
||||
subject = x509.Name([
|
||||
x509.NameAttribute(NameOID.COMMON_NAME, domain),
|
||||
])
|
||||
now = datetime.datetime.now(datetime.timezone.utc)
|
||||
cert = (
|
||||
x509.CertificateBuilder()
|
||||
.subject_name(subject)
|
||||
.issuer_name(self._ca_cert.subject)
|
||||
.public_key(key.public_key())
|
||||
.serial_number(x509.random_serial_number())
|
||||
.not_valid_before(now)
|
||||
.not_valid_after(now + datetime.timedelta(days=365))
|
||||
.add_extension(
|
||||
x509.SubjectAlternativeName([x509.DNSName(domain)]),
|
||||
critical=False,
|
||||
)
|
||||
.sign(self._ca_key, hashes.SHA256())
|
||||
)
|
||||
|
||||
key_pem = key.private_bytes(
|
||||
serialization.Encoding.PEM,
|
||||
serialization.PrivateFormat.TraditionalOpenSSL,
|
||||
serialization.NoEncryption(),
|
||||
)
|
||||
cert_pem = cert.public_bytes(serialization.Encoding.PEM)
|
||||
return key_pem, cert_pem
|
||||
+520
@@ -0,0 +1,520 @@
|
||||
"""
|
||||
Local HTTP proxy server.
|
||||
|
||||
Intercepts the user's browser traffic and forwards everything through
|
||||
a domain-fronted connection to a CDN worker or Apps Script relay.
|
||||
|
||||
Supports:
|
||||
- CONNECT method → WebSocket tunnel (modes 1-3) or MITM relay (apps_script)
|
||||
- GET / POST etc. → HTTP forwarding (modes 1-3) or JSON relay (apps_script)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
|
||||
from domain_fronter import DomainFronter
|
||||
|
||||
log = logging.getLogger("Proxy")
|
||||
|
||||
|
||||
class ResponseCache:
|
||||
"""Simple LRU response cache — avoids repeated relay calls."""
|
||||
|
||||
def __init__(self, max_mb: int = 50):
|
||||
self._store: dict[str, tuple[bytes, float]] = {}
|
||||
self._size = 0
|
||||
self._max = max_mb * 1024 * 1024
|
||||
self.hits = 0
|
||||
self.misses = 0
|
||||
|
||||
def get(self, url: str) -> bytes | None:
|
||||
entry = self._store.get(url)
|
||||
if not entry:
|
||||
self.misses += 1
|
||||
return None
|
||||
raw, expires = entry
|
||||
if time.time() > expires:
|
||||
self._size -= len(raw)
|
||||
del self._store[url]
|
||||
self.misses += 1
|
||||
return None
|
||||
self.hits += 1
|
||||
return raw
|
||||
|
||||
def put(self, url: str, raw_response: bytes, ttl: int = 300):
|
||||
size = len(raw_response)
|
||||
if size > self._max // 4 or size == 0:
|
||||
return
|
||||
# Evict oldest to make room
|
||||
while self._size + size > self._max and self._store:
|
||||
oldest = next(iter(self._store))
|
||||
self._size -= len(self._store[oldest][0])
|
||||
del self._store[oldest]
|
||||
if url in self._store:
|
||||
self._size -= len(self._store[url][0])
|
||||
self._store[url] = (raw_response, time.time() + ttl)
|
||||
self._size += size
|
||||
|
||||
@staticmethod
|
||||
def parse_ttl(raw_response: bytes, url: str) -> int:
|
||||
"""Determine cache TTL from response headers and URL."""
|
||||
hdr_end = raw_response.find(b"\r\n\r\n")
|
||||
if hdr_end < 0:
|
||||
return 0
|
||||
hdr = raw_response[:hdr_end].decode(errors="replace").lower()
|
||||
|
||||
# Don't cache errors or non-200
|
||||
if b"HTTP/1.1 200" not in raw_response[:20]:
|
||||
return 0
|
||||
if "no-store" in hdr:
|
||||
return 0
|
||||
|
||||
# Explicit max-age
|
||||
m = re.search(r"max-age=(\d+)", hdr)
|
||||
if m:
|
||||
return min(int(m.group(1)), 86400)
|
||||
|
||||
# Heuristic by content type / extension
|
||||
path = url.split("?")[0].lower()
|
||||
static_exts = (
|
||||
".css", ".js", ".woff", ".woff2", ".ttf", ".eot",
|
||||
".png", ".jpg", ".jpeg", ".gif", ".webp", ".svg", ".ico",
|
||||
".mp3", ".mp4", ".wasm",
|
||||
)
|
||||
for ext in static_exts:
|
||||
if path.endswith(ext):
|
||||
return 3600 # 1 hour for static assets
|
||||
|
||||
ct_m = re.search(r"content-type:\s*([^\r\n]+)", hdr)
|
||||
ct = ct_m.group(1) if ct_m else ""
|
||||
if "image/" in ct or "font/" in ct:
|
||||
return 3600
|
||||
if "text/css" in ct or "javascript" in ct:
|
||||
return 1800
|
||||
if "text/html" in ct or "application/json" in ct:
|
||||
return 0 # don't cache dynamic content by default
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
class ProxyServer:
|
||||
def __init__(self, config: dict):
|
||||
self.host = config.get("listen_host", "127.0.0.1")
|
||||
self.port = config.get("listen_port", 8080)
|
||||
self.mode = config.get("mode", "domain_fronting")
|
||||
self.fronter = DomainFronter(config)
|
||||
self.mitm = None
|
||||
self._cache = ResponseCache(max_mb=50)
|
||||
|
||||
# Persistent HTTP tunnel cache for google_fronting mode
|
||||
# Key: "host:port" → (tunnel_reader, tunnel_writer, lock)
|
||||
self._http_tunnels: dict = {}
|
||||
self._tunnel_lock = asyncio.Lock()
|
||||
|
||||
if self.mode == "apps_script":
|
||||
try:
|
||||
from mitm import MITMCertManager
|
||||
self.mitm = MITMCertManager()
|
||||
except ImportError:
|
||||
log.error("apps_script mode requires 'cryptography' package.")
|
||||
log.error("Run: pip install cryptography")
|
||||
raise SystemExit(1)
|
||||
|
||||
async def start(self):
|
||||
srv = await asyncio.start_server(self._on_client, self.host, self.port)
|
||||
log.info(
|
||||
"Listening on %s:%d — configure your browser HTTP proxy to this address",
|
||||
self.host, self.port,
|
||||
)
|
||||
async with srv:
|
||||
await srv.serve_forever()
|
||||
|
||||
# ── client handler ────────────────────────────────────────────
|
||||
|
||||
async def _on_client(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter):
|
||||
addr = writer.get_extra_info("peername")
|
||||
try:
|
||||
first_line = await asyncio.wait_for(reader.readline(), timeout=30)
|
||||
if not first_line:
|
||||
return
|
||||
|
||||
# Read remaining headers
|
||||
header_block = first_line
|
||||
while True:
|
||||
line = await asyncio.wait_for(reader.readline(), timeout=10)
|
||||
header_block += line
|
||||
if line in (b"\r\n", b"\n", b""):
|
||||
break
|
||||
|
||||
request_line = first_line.decode(errors="replace").strip()
|
||||
parts = request_line.split(" ", 2)
|
||||
if len(parts) < 2:
|
||||
return
|
||||
|
||||
method = parts[0].upper()
|
||||
|
||||
if method == "CONNECT":
|
||||
await self._do_connect(parts[1], reader, writer)
|
||||
else:
|
||||
await self._do_http(header_block, reader, writer)
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
log.debug("Timeout: %s", addr)
|
||||
except Exception as e:
|
||||
log.error("Error (%s): %s", addr, e)
|
||||
finally:
|
||||
try:
|
||||
writer.close()
|
||||
await writer.wait_closed()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ── CONNECT (HTTPS tunnelling) ────────────────────────────────
|
||||
|
||||
async def _do_connect(self, target: str, reader, writer):
|
||||
host, _, port = target.rpartition(":")
|
||||
port = int(port) if port else 443
|
||||
if not host:
|
||||
host, port = target, 443
|
||||
|
||||
log.info("CONNECT → %s:%d", host, port)
|
||||
|
||||
writer.write(b"HTTP/1.1 200 Connection Established\r\n\r\n")
|
||||
await writer.drain()
|
||||
|
||||
if self.mode == "apps_script":
|
||||
# Google services: tunnel directly (no MITM) to avoid
|
||||
# Google's anti-bot detection from Apps Script IPs/UA.
|
||||
if self._is_google_domain(host):
|
||||
log.info("Direct tunnel → %s (Google domain, skipping relay)", host)
|
||||
await self._do_direct_tunnel(host, port, reader, writer)
|
||||
else:
|
||||
await self._do_mitm_connect(host, port, reader, writer)
|
||||
else:
|
||||
await self.fronter.tunnel(host, port, reader, writer)
|
||||
|
||||
# ── Google domain detection ───────────────────────────────────
|
||||
|
||||
# Only domains whose SNI the ISP does NOT block.
|
||||
# YouTube/googlevideo are blocked by SNI inspection in Iran,
|
||||
# so they MUST go through the MITM relay (domain-fronted).
|
||||
_GOOGLE_SUFFIXES = (
|
||||
".google.com", ".google.co",
|
||||
".googleapis.com", ".gstatic.com",
|
||||
".googleusercontent.com",
|
||||
)
|
||||
_GOOGLE_EXACT = {
|
||||
"google.com", "gstatic.com", "googleapis.com",
|
||||
}
|
||||
|
||||
def _is_google_domain(self, host: str) -> bool:
|
||||
"""Return True if host is a Google-owned domain."""
|
||||
h = host.lower().rstrip(".")
|
||||
if h in self._GOOGLE_EXACT:
|
||||
return True
|
||||
for suffix in self._GOOGLE_SUFFIXES:
|
||||
if h.endswith(suffix):
|
||||
return True
|
||||
return False
|
||||
|
||||
# ── Direct tunnel (no MITM) ───────────────────────────────────
|
||||
|
||||
async def _do_direct_tunnel(self, host: str, port: int,
|
||||
reader: asyncio.StreamReader,
|
||||
writer: asyncio.StreamWriter):
|
||||
"""Pipe raw TLS bytes directly to the target server.
|
||||
|
||||
Used for Google domains: the browser's TLS goes end-to-end
|
||||
with Google, preserving real User-Agent and avoiding
|
||||
Apps Script IP/bot-detection issues.
|
||||
"""
|
||||
google_ip = self.fronter.connect_host
|
||||
try:
|
||||
r_remote, w_remote = await asyncio.wait_for(
|
||||
asyncio.open_connection(google_ip, port), timeout=10
|
||||
)
|
||||
except Exception as e:
|
||||
log.error("Direct tunnel connect failed (%s via %s): %s",
|
||||
host, google_ip, e)
|
||||
return
|
||||
|
||||
async def pipe(src, dst, label):
|
||||
try:
|
||||
while True:
|
||||
data = await src.read(65536)
|
||||
if not data:
|
||||
break
|
||||
dst.write(data)
|
||||
await dst.drain()
|
||||
except (ConnectionError, asyncio.CancelledError):
|
||||
pass
|
||||
except Exception as e:
|
||||
log.debug("Pipe %s ended: %s", label, e)
|
||||
finally:
|
||||
try:
|
||||
dst.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
await asyncio.gather(
|
||||
pipe(reader, w_remote, f"client→{host}"),
|
||||
pipe(r_remote, writer, f"{host}→client"),
|
||||
)
|
||||
|
||||
# ── MITM CONNECT (apps_script mode) ───────────────────────────
|
||||
|
||||
async def _do_mitm_connect(self, host: str, port: int, reader, writer):
|
||||
"""Intercept TLS, decrypt HTTP, and relay through Apps Script."""
|
||||
ssl_ctx = self.mitm.get_server_context(host)
|
||||
|
||||
# Upgrade the existing connection to TLS (we are the server)
|
||||
loop = asyncio.get_event_loop()
|
||||
transport = writer.transport
|
||||
protocol = transport.get_protocol()
|
||||
|
||||
try:
|
||||
new_transport = await loop.start_tls(
|
||||
transport, protocol, ssl_ctx, server_side=True,
|
||||
)
|
||||
except Exception as e:
|
||||
log.error("TLS handshake failed for %s: %s", host, e)
|
||||
return
|
||||
|
||||
# Update writer to use the new TLS transport
|
||||
writer._transport = new_transport
|
||||
|
||||
# Read and relay HTTP requests from the browser (now decrypted)
|
||||
while True:
|
||||
try:
|
||||
first_line = await asyncio.wait_for(reader.readline(), timeout=120)
|
||||
if not first_line:
|
||||
break
|
||||
|
||||
header_block = first_line
|
||||
while True:
|
||||
line = await asyncio.wait_for(reader.readline(), timeout=10)
|
||||
header_block += line
|
||||
if line in (b"\r\n", b"\n", b""):
|
||||
break
|
||||
|
||||
# Read body
|
||||
body = b""
|
||||
for raw_line in header_block.split(b"\r\n"):
|
||||
if raw_line.lower().startswith(b"content-length:"):
|
||||
length = int(raw_line.split(b":", 1)[1].strip())
|
||||
body = await reader.readexactly(length)
|
||||
break
|
||||
|
||||
# Parse the request
|
||||
request_line = first_line.decode(errors="replace").strip()
|
||||
parts = request_line.split(" ", 2)
|
||||
if len(parts) < 2:
|
||||
break
|
||||
|
||||
method = parts[0]
|
||||
path = parts[1]
|
||||
|
||||
# Parse headers
|
||||
headers = {}
|
||||
for raw_line in header_block.split(b"\r\n")[1:]:
|
||||
if b":" in raw_line:
|
||||
k, v = raw_line.decode(errors="replace").split(":", 1)
|
||||
headers[k.strip()] = v.strip()
|
||||
|
||||
# Build full URL (browser sends just the path in CONNECT)
|
||||
if port == 443:
|
||||
url = f"https://{host}{path}"
|
||||
else:
|
||||
url = f"https://{host}:{port}{path}"
|
||||
|
||||
log.info("MITM → %s %s", method, url)
|
||||
|
||||
# Check local cache first (GET only)
|
||||
response = None
|
||||
if method == "GET" and not body:
|
||||
response = self._cache.get(url)
|
||||
if response:
|
||||
log.debug("Cache HIT: %s", url[:60])
|
||||
|
||||
if response is None:
|
||||
# Relay through Apps Script
|
||||
try:
|
||||
response = await self._relay_smart(method, url, headers, body)
|
||||
except Exception as e:
|
||||
log.error("Relay error (%s): %s", url[:60], e)
|
||||
err_body = f"Relay error: {e}".encode()
|
||||
response = (
|
||||
b"HTTP/1.1 502 Bad Gateway\r\n"
|
||||
b"Content-Type: text/plain\r\n"
|
||||
b"Content-Length: " + str(len(err_body)).encode() + b"\r\n"
|
||||
b"\r\n" + err_body
|
||||
)
|
||||
|
||||
# Cache successful GET responses
|
||||
if method == "GET" and not body and response:
|
||||
ttl = ResponseCache.parse_ttl(response, url)
|
||||
if ttl > 0:
|
||||
self._cache.put(url, response, ttl)
|
||||
log.debug("Cached (%ds): %s", ttl, url[:60])
|
||||
|
||||
writer.write(response)
|
||||
await writer.drain()
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
break
|
||||
except asyncio.IncompleteReadError:
|
||||
break
|
||||
except ConnectionError:
|
||||
break
|
||||
except Exception as e:
|
||||
log.error("MITM handler error (%s): %s", host, e)
|
||||
break
|
||||
|
||||
async def _relay_smart(self, method, url, headers, body):
|
||||
"""Choose optimal relay strategy based on request type.
|
||||
|
||||
ALL GET requests go through relay_parallel: it does one probe
|
||||
request and only splits into parallel chunks if the response
|
||||
is large and the server supports ranges. Small responses still
|
||||
use a single request (no overhead).
|
||||
"""
|
||||
if method == "GET" and not body:
|
||||
# Skip parallel-range if the client already sent a Range header
|
||||
# (we must forward it verbatim, not modify it).
|
||||
if headers:
|
||||
for k in headers:
|
||||
if k.lower() == "range":
|
||||
return await self.fronter.relay(
|
||||
method, url, headers, body
|
||||
)
|
||||
return await self.fronter.relay_parallel(
|
||||
method, url, headers, body
|
||||
)
|
||||
return await self.fronter.relay(method, url, headers, body)
|
||||
|
||||
def _is_likely_download(self, url: str, headers: dict) -> bool:
|
||||
"""Heuristic: is this URL likely a large file download?"""
|
||||
# Check file extension
|
||||
path = url.split("?")[0].lower()
|
||||
large_exts = {
|
||||
".zip", ".tar", ".gz", ".bz2", ".xz", ".7z", ".rar",
|
||||
".exe", ".msi", ".dmg", ".deb", ".rpm", ".apk",
|
||||
".iso", ".img",
|
||||
".mp4", ".mkv", ".avi", ".mov", ".webm",
|
||||
".mp3", ".flac", ".wav", ".aac",
|
||||
".pdf", ".doc", ".docx", ".ppt", ".pptx",
|
||||
".wasm",
|
||||
}
|
||||
for ext in large_exts:
|
||||
if path.endswith(ext):
|
||||
return True
|
||||
return False
|
||||
|
||||
# ── Plain HTTP forwarding ─────────────────────────────────────
|
||||
|
||||
async def _do_http(self, header_block: bytes, reader, writer):
|
||||
body = b""
|
||||
for raw_line in header_block.split(b"\r\n"):
|
||||
if raw_line.lower().startswith(b"content-length:"):
|
||||
length = int(raw_line.split(b":", 1)[1].strip())
|
||||
body = await reader.readexactly(length)
|
||||
break
|
||||
|
||||
first_line = header_block.split(b"\r\n")[0].decode(errors="replace")
|
||||
log.info("HTTP → %s", first_line)
|
||||
|
||||
if self.mode == "apps_script":
|
||||
# Parse request and relay through Apps Script
|
||||
parts = first_line.strip().split(" ", 2)
|
||||
method = parts[0] if parts else "GET"
|
||||
url = parts[1] if len(parts) > 1 else "/"
|
||||
|
||||
headers = {}
|
||||
for raw_line in header_block.split(b"\r\n")[1:]:
|
||||
if b":" in raw_line:
|
||||
k, v = raw_line.decode(errors="replace").split(":", 1)
|
||||
headers[k.strip()] = v.strip()
|
||||
|
||||
# Cache check for GET
|
||||
response = None
|
||||
if method == "GET" and not body:
|
||||
response = self._cache.get(url)
|
||||
if response:
|
||||
log.debug("Cache HIT (HTTP): %s", url[:60])
|
||||
|
||||
if response is None:
|
||||
response = await self._relay_smart(method, url, headers, body)
|
||||
# Cache successful GET
|
||||
if method == "GET" and not body and response:
|
||||
ttl = ResponseCache.parse_ttl(response, url)
|
||||
if ttl > 0:
|
||||
self._cache.put(url, response, ttl)
|
||||
elif self.mode in ("google_fronting", "custom_domain", "domain_fronting"):
|
||||
# Use WebSocket tunnel for ALL traffic (much faster than forward())
|
||||
response = await self._tunnel_http(header_block, body)
|
||||
else:
|
||||
response = await self.fronter.forward(header_block + body)
|
||||
|
||||
writer.write(response)
|
||||
await writer.drain()
|
||||
|
||||
async def _tunnel_http(self, header_block: bytes, body: bytes) -> bytes:
|
||||
"""Forward plain HTTP via a persistent WebSocket tunnel.
|
||||
|
||||
Instead of opening a new TLS+HTTP connection for each request
|
||||
(the old forward() path), this keeps a WebSocket tunnel open
|
||||
to the target host and pipes raw HTTP through it.
|
||||
Much faster for rapid-fire requests (e.g., Telegram API).
|
||||
"""
|
||||
import re as _re
|
||||
|
||||
# Parse target host:port from the raw HTTP request
|
||||
host = ""
|
||||
port = 80
|
||||
for line in header_block.split(b"\r\n")[1:]:
|
||||
if not line:
|
||||
break
|
||||
if line.lower().startswith(b"host:"):
|
||||
host_val = line.split(b":", 1)[1].strip().decode(errors="replace")
|
||||
if ":" in host_val:
|
||||
h, p = host_val.rsplit(":", 1)
|
||||
try:
|
||||
host, port = h, int(p)
|
||||
except ValueError:
|
||||
host = host_val
|
||||
else:
|
||||
host = host_val
|
||||
break
|
||||
|
||||
if not host:
|
||||
return b"HTTP/1.1 400 Bad Request\r\n\r\nNo Host header\r\n"
|
||||
|
||||
# Rewrite the request line: browser sends absolute URL
|
||||
# (e.g., "GET http://host/path HTTP/1.1") but the target
|
||||
# server expects a relative path ("GET /path HTTP/1.1")
|
||||
first_line = header_block.split(b"\r\n")[0]
|
||||
first_str = first_line.decode(errors="replace")
|
||||
parts = first_str.split(" ", 2)
|
||||
if len(parts) >= 2 and parts[1].startswith("http://"):
|
||||
from urllib.parse import urlparse
|
||||
parsed = urlparse(parts[1])
|
||||
rel_path = parsed.path or "/"
|
||||
if parsed.query:
|
||||
rel_path += "?" + parsed.query
|
||||
new_first = f"{parts[0]} {rel_path}"
|
||||
if len(parts) == 3:
|
||||
new_first += f" {parts[2]}"
|
||||
header_block = new_first.encode() + b"\r\n" + b"\r\n".join(header_block.split(b"\r\n")[1:])
|
||||
|
||||
raw_request = header_block + body
|
||||
|
||||
# Send through tunnel
|
||||
try:
|
||||
return await asyncio.wait_for(
|
||||
self.fronter.forward(raw_request), timeout=30
|
||||
)
|
||||
except Exception as e:
|
||||
log.error("Tunnel HTTP failed (%s:%d): %s", host, port, e)
|
||||
return b"HTTP/1.1 502 Bad Gateway\r\n\r\nTunnel forward failed\r\n"
|
||||
@@ -0,0 +1,8 @@
|
||||
# Core (no external dependencies for basic modes)
|
||||
# Python 3.10+ required
|
||||
|
||||
# Optional: MITM interception (apps_script mode)
|
||||
cryptography>=41.0.0
|
||||
|
||||
# Optional: HTTP/2 multiplexing (faster apps_script relay)
|
||||
h2>=4.1.0
|
||||
@@ -0,0 +1,76 @@
|
||||
"""
|
||||
Minimal WebSocket frame encoder / decoder (RFC 6455).
|
||||
|
||||
Only handles binary (opcode 0x02) and close (opcode 0x08) frames.
|
||||
Client-to-server frames are always masked as required by the spec.
|
||||
"""
|
||||
|
||||
import os
|
||||
import struct
|
||||
|
||||
|
||||
def ws_encode(data: bytes, opcode: int = 0x02) -> bytes:
|
||||
"""Encode *data* into a masked binary WebSocket frame."""
|
||||
head = bytearray([0x80 | opcode]) # FIN + opcode
|
||||
|
||||
length = len(data)
|
||||
if length < 126:
|
||||
head.append(0x80 | length)
|
||||
elif length < 0x10000:
|
||||
head.append(0x80 | 126)
|
||||
head += struct.pack("!H", length)
|
||||
else:
|
||||
head.append(0x80 | 127)
|
||||
head += struct.pack("!Q", length)
|
||||
|
||||
mask = os.urandom(4)
|
||||
head += mask
|
||||
|
||||
masked = bytearray(data)
|
||||
for i in range(len(masked)):
|
||||
masked[i] ^= mask[i & 3]
|
||||
|
||||
return bytes(head) + bytes(masked)
|
||||
|
||||
|
||||
def ws_decode(buf: bytes):
|
||||
"""Try to decode one frame from *buf*.
|
||||
|
||||
Returns ``(opcode, payload, consumed_bytes)`` or ``None`` if the
|
||||
buffer does not yet contain a complete frame.
|
||||
"""
|
||||
if len(buf) < 2:
|
||||
return None
|
||||
|
||||
opcode = buf[0] & 0x0F
|
||||
is_masked = buf[1] & 0x80
|
||||
length = buf[1] & 0x7F
|
||||
pos = 2
|
||||
|
||||
if length == 126:
|
||||
if len(buf) < 4:
|
||||
return None
|
||||
length = struct.unpack("!H", buf[2:4])[0]
|
||||
pos = 4
|
||||
elif length == 127:
|
||||
if len(buf) < 10:
|
||||
return None
|
||||
length = struct.unpack("!Q", buf[2:10])[0]
|
||||
pos = 10
|
||||
|
||||
mask = None
|
||||
if is_masked:
|
||||
if len(buf) < pos + 4:
|
||||
return None
|
||||
mask = buf[pos : pos + 4]
|
||||
pos += 4
|
||||
|
||||
if len(buf) < pos + length:
|
||||
return None
|
||||
|
||||
payload = bytearray(buf[pos : pos + length])
|
||||
if mask:
|
||||
for i in range(len(payload)):
|
||||
payload[i] ^= mask[i & 3]
|
||||
|
||||
return opcode, bytes(payload), pos + length
|
||||
Reference in New Issue
Block a user