Freelance AI Infrastructure Engineer

Scope: Set up and configure a local AI server for a civil engineering firm on a dedicated Windows workstation (92GB GPU VRAM). The stack includes WSL2, Docker, vLLM serving a 70B-parameter model (Qwen2.5-72B or Llama 3.3-70B), and AnythingLLM as a RAG document assistant. Internal network deployment for team access also required.

Deliverables:

• vLLM running reliably via Docker Compose with auto-restart
• AnythingLLM connected to local model and indexed with firm documents
• Internal network access configured for team
• Basic documentation of the setup for internal IT reference

Skills required: Linux/WSL2, Docker, NVIDIA CUDA, vLLM or similar LLM serving, RAG pipelines

Estimated effort: 2-5 days. Open to hourly or fixed-price proposals.

Note: All work involves sensitive engineering project data — candidate must be comfortable working within a fully on-premises, no-cloud-data-transfer constraint. This position is not eligible for remote work. The candidate must be local to Mason, OH.

Work Location: In person