Domain Specialization (LoRA + RAG)

Published 2026-04-04·Updated 2026-10-29·23 min read

AI documentation guide

Provides a 3-stage strategy for optimizing general-purpose LLMs for specific domains such as finance, telecommunications, and manufacturing to improve coding quality.

Core Question

"Why doesn't code generated by Claude or GPT follow our company standards?" → Because the model hasn't learned your domain knowledge.

---

3-Layer Strategy

Apply domain specialization in three stages: Steering → RAG → LoRA. Steering supplies explicit coding instructions; RAG retrieves relevant internal documents for the prompt; LoRA adapts the model through training on domain data. The sections below explain what each stage adds.

Figure 1. Domain specialization 3-layer application order and cost

Expand the diagram to read labels and follow connections.

Diagram

Layer 1: Steering (Immediate)

Definition: Explicitly define coding rules in spec files to instruct the LLM.

Pros:

• Immediately applicable
• Zero cost
• Easy maintenance (just edit spec files)

Cons:

• Limited for complex domain logic
• Context window waste

Example:

# coding-standards.md

## Coding Conventions
- Class names: PascalCase
- Method names: camelCase
- Constants: UPPER_SNAKE_CASE

## Transaction Handling
- All DB operations must use @Transactional
- Rollback condition: on RuntimeException

## Logging Standards
- Entry point: log.info("Method {} started", methodName)
- Exceptions: log.error("Error in {}: {}", methodName, e.getMessage())

Layer 2: RAG (1-2 weeks)

Definition: Embed internal documents in a vector DB for real-time retrieval, including relevant information in prompts.

Pros:

• Auto-reflects latest documents (no retraining)
• High accuracy for internal API specs
• No model weight changes

Cons:

• Infrastructure required (Milvus, Neo4j)
• Retrieval quality directly impacts output quality
• Embedding costs

Example:

from langchain_milvus import Milvus
from langchain_openai import OpenAIEmbeddings

# 1. Embed internal API documentation
embeddings = OpenAIEmbeddings()
vectorstore = Milvus.from_documents(
    documents=internal_api_docs,
    embedding=embeddings,
    connection_args={"host": "milvus.cluster.local", "port": 19530}
)

# 2. Keep the index fresh by updating/deleting documents
#    - For changed docs, delete by doc_id (pk) and re-insert to avoid duplicates.
vectorstore.delete(ids=changed_doc_ids)
vectorstore.add_documents(updated_api_docs)
#    - Remove deleted source documents from the index as well.
vectorstore.delete(ids=removed_doc_ids)

# 3. Search for relevant documents
query = "How to call user authentication API?"
docs = vectorstore.similarity_search(query, k=3)

# 4. Pass search results + query to LLM
prompt = f"Context: {docs}\n\nQuestion: {query}"

If you only add documents and never reflect changes or deletions, the index returns stale content, so operate update, deletion, and index-freshness maintenance alongside ingestion.

Layer 3: LoRA (1-2 months)

Definition: Adjust model weights with domain data to generate domain expert-level output.

Pros: Consistent code style, highest domain terminology accuracy, complex pattern learning Cons: GPU training cost ($2,000), training data collection required

Kiro GLM-5 vs Self-Hosting

Kiro IDE has natively supported GLM-5 since April 2026 and is immediately available. However, LoRA Fine-tuning, Multi-LoRA hot-swap for multiple customers, and self-controlled compliance are only possible with self-hosting. Recommendation: Kiro for prototyping, self-hosting for production domain specialization

For detailed LoRA training and deployment pipeline implementation, see Custom Model Pipeline — LoRA Training & Deployment Pipeline (Domain Specialization). Includes QLoRA GPU optimization, training data format, NeMo/Unsloth frameworks, checkpoint management, and Multi-LoRA hot-swap deployment configuration.

---

Required Layers by Scenario

The suitability ratings and the cost-vs-effect figures in the table below are not measurements but illustrative planning assumptions to help choose layers. Actual suitability and improvement depend on the target task and the controls being assessed, so measure them in your own environment before adoption.

Table 1. Suitability of each layer and recommended combination by requirement (illustrative assumptions)

Requirement	Layer 1 (Steering)	Layer 2 (RAG)	Layer 3 (LoRA)	Recommended Combination
Coding conventions	Sufficient	Excessive	Unnecessary	Layer 1
Internal API usage	Insufficient	Required	Unnecessary	Layer 1 + 2
Domain terminology	Limited	Supplementary	Required	Layer 2 + 3
SOC2 procedures	Playbook sufficient	Unnecessary	Unnecessary	Layer 1
Consistent code style	Basic only	Supplementary	Most effective	Layer 1 + 3
Legacy migration patterns	Impossible	Example provision	Core	Layer 2 + 3

Cost vs Effect (illustrative assumptions, not measured)

The improvement figures are not measured results for a specific task or dataset; they are assumptions for comparing layer combinations.

• Layer 1 only: Free, illustrative improvement 60%
• Layer 1 + 2: Infrastructure cost, illustrative improvement 80%
• Layer 1 + 2 + 3: $2,000, illustrative improvement 95%

---

VectorRAG Configuration

VectorRAG is a document retrieval-based domain specialization approach.

Architecture

Figure 2. VectorRAG architecture

Expand the diagram to read labels and follow connections.

Diagram

Knowledge Feature Store Integration

Integrates with Layer 5: Knowledge Feature Store of the LG U+ Agentic AI Platform for vector search.

apiVersion: feast.dev/v1
kind: FeatureStore
metadata:
  name: knowledge-feature-store
  namespace: ai-platform
spec:
  feastProject: knowledge_feature_store
  services:
    onlineStore:
      persistence:
        store:
          type: milvus
          secretRef:
            name: milvus-connection
---
# Manage Milvus connection details via a Secret (feature_store.yaml format)
apiVersion: v1
kind: Secret
metadata:
  name: milvus-connection
  namespace: ai-platform
type: Opaque
stringData:
  feature_store.yaml: |
    project: knowledge_feature_store
    provider: local
    online_store:
      type: milvus
      host: milvus.cluster.local
      port: 19530

Entities and features are defined in a Feast feature repo (Python definitions + feast apply), not in the CR:

# features.py - deployed with feast apply
from feast import Entity, FeatureView, Field
from feast.types import String, Array, Float64

# Entity definition
api_doc = Entity(
    name="api_doc",
    join_keys=["doc_id"],
    value_type=String
)

# Feature definition
api_embedding_view = FeatureView(
    name="api_embeddings",
    entities=[api_doc],
    schema=[
        Field(name="api_embedding", dtype=Array(Float64)),
    ],
)

Data Flow

1. Document Collection: Confluence, GitHub, Wiki → Crawling
2. Chunk Splitting: Split into 512-token chunks (50-token overlap)
3. Embedding: OpenAI text-embedding-3-large or BGE-M3
4. Vector Storage: Store in Milvus collection
5. Search: Question embedding → Cosine similarity Top-K
6. LLM Delivery: Search results + question → LLM

Chunk Size Optimization

• Too small: Context loss
• Too large: Noise increase
• Recommended: 512 tokens, 50-token overlap

---

GraphRAG Configuration

GraphRAG is a knowledge graph-based domain specialization approach. It explicitly models relationships between financial business terminology and regulations.

Architecture

Figure 3. GraphRAG architecture

Expand the diagram to read labels and follow connections.

Diagram

Ontology-Based Structure

Defines entities, relations, and attributes in the financial domain.

// Entity definitions
CREATE (loan:Product {name: "Mortgage Loan", type: "Loan"})
CREATE (credit:Criteria {name: "Credit Score", threshold: 600})
CREATE (reg:Regulation {code: "Banking Supervision Regulation Article 35"})

// Relationship definitions
CREATE (loan)-[:REQUIRES]->(credit)
CREATE (loan)-[:GOVERNED_BY]->(reg)
CREATE (credit)-[:VERIFIED_BY]->(cbService:Service {name: "CB Inquiry"})

VectorRAG + GraphRAG Hybrid

Figure 4. VectorRAG + GraphRAG hybrid configuration

Expand the diagram to read labels and follow connections.

Diagram

Advantages:

• VectorRAG: Reflects latest documents
• GraphRAG: Complex rule reasoning
• Hybrid: Accuracy + Flexibility

Production Example

Question: "Can a customer with credit score 550 get a mortgage loan?"

1. VectorRAG: Search "mortgage loan" documents → "Credit score 600+ required"
2. GraphRAG: Traverse (loan)-[:REQUIRES]->(credit {threshold: 600})
3. LLM Judgment: "550 < 600 → Not eligible" + "Credit score improvement guidance"

---

FSI SI Production Scenarios

Scenario 1: COBOL → Java Legacy Migration

Effect Comparison by Layer

Table 2. Accuracy, consistency, and cost comparison by layer for COBOL→Java migration

Approach	Accuracy	Consistency	Cost	Notes
Steering only	60%	Low	Free	Syntax correct but financial logic errors
+ RAG	80%	Medium	Infrastructure	Improved accuracy, inconsistent patterns
+ LoRA	95%	High	$2,000	Consistent patterns + financial logic

ROI Analysis

Assumptions:

• 10,000 modules to migrate
• Developer hourly rate: $50/hr

Table 3. Time per module, total time, and total cost by migration method (assumption-based estimate)

Method	Time/Module	Total Time	Total Cost	Notes
Manual	2 hours	20,000 hrs	$1,000,000	-
LLM (Steering+RAG)	1 hour	10,000 hrs	$500,000	Savings: $500,000
LLM (+ LoRA)	30 min	5,000 hrs	$250,000 + $2,000	Savings: $748,000

ROI:

• LoRA training cost (investment denominator): $2,000
• Cost savings (vs. baseline): $748,000
• ROI: 374x (hypothetical, not an observed LoRA ROI)

This figure is a hypothetical calculation based on the assumptions above (10,000 modules, $50/hr, time per module); it is not a measured result. The baseline is "manual migration," and the investment denominator includes only the $2,000 LoRA training cost. Inference costs, operations costs, and review/rework costs for the output are excluded from the denominator, so the real ROI is lower. The 95% accuracy is also an assumption.

Production Example

The COBOL and Java below are an illustrative example (pseudocode) showing a conversion result; they are not a conversion verified to preserve behavior for a specific COBOL dialect. Because the COBOL fragment is incomplete, a real migration must specify the target dialect and the complete behavior contract and then verify it.

Input (COBOL):

PERFORM CALC-INTEREST
    USING WS-PRINCIPAL WS-RATE
    GIVING WS-INTEREST.
IF WS-CREDIT-SCORE < 600
    MOVE 'REJECT' TO WS-RESULT
ELSE
    MOVE 'APPROVE' TO WS-RESULT.

Output (Java, after LoRA training):

@Service
@Transactional
public class LoanService {
    
    @AuditLog(regulation = "Banking Supervision Regulation Article 35")
    public LoanDecision processLoan(BigDecimal principal, BigDecimal rate, int creditScore) {
        BigDecimal interest = calcInterest(principal, rate);
        
        if (creditScore < 600) {
            return LoanDecision.REJECT;
        }
        return LoanDecision.APPROVE;
    }
    
    private BigDecimal calcInterest(BigDecimal principal, BigDecimal rate) {
        return principal.multiply(rate).setScale(2, RoundingMode.HALF_UP);
    }
}

---

Scenario 2: Internal Framework Code Generation

In SI environments using proprietary frameworks (Samsung SDS Devon, LG CNS Anyframe, etc.), general-purpose LLMs cannot generate accurate code.

Solution

1. LoRA learns framework patterns

   {"input": "Create user lookup API", "output": "@DevonController\npublic class UserController extends AbstractController {\n    @DevonService\n    private UserService userService;\n    ..."}

2. RAG searches framework API documentation

   # Embed Devon API documentation (use add_texts for strings, add_documents for Document objects)
   texts = ["DevonController usage", "DevonService transaction handling", ...]
   vectorstore.add_texts(texts)

3. Steering enforces conventions

   - All Controllers must extend AbstractController
   - Services must use @DevonService annotation

Effect

The figures below are not measurements but illustrative adoption targets.

• Internal framework code generation accuracy: target example 95%
• Junior developer onboarding time: example 3 months → 1 month

---

Scenario 3: Regulatory Compliance Code Auto-Generation

Automatically reflects financial regulations (Electronic Financial Supervisory Regulation (전자금융감독규정), Banking Supervision Regulation (은행업감독규정)) into code.

The training data and generated result below are a synthetic example that shows how regulatory requirements can be marked with annotations and comments. The article numbers and the annotation mapping in the example have not been verified against the current text of the regulations, and the generated code does not guarantee regulatory compliance. Before applying this pattern, legal and compliance staff must review the current articles and the mapping, and the controls the annotations point to (audit logging, access control, identity verification) must be separately validated as actually implemented, authenticated, and working.

Training Data Example

{"input": "Loan approval API", "output": "@AuditLog(regulation = \"Banking Supervision Regulation Article 35\")\n@AccessControl(level = AccessLevel.CRITICAL)\npublic LoanDecision approveLoan(LoanRequest req) {\n    validateTransactionLimit(req); // Article 34\n    fdsService.checkAnomalySync(req); // FDS integration\n    ...\n}"}

Auto-Generated Result

@RestController
@RequestMapping("/api/loan")
public class LoanController {
    
    @AuditLog(regulation = "Banking Supervision Regulation Article 35")
    @AccessControl(level = AccessLevel.CRITICAL)
    @PostMapping("/approve")
    public LoanResponse approveLoan(@RequestBody LoanRequest req) {
        // Article 34: Transaction limit validation
        validateTransactionLimit(req);
        
        // FDS anomaly detection (Article 15)
        if (fdsService.detectAnomaly(req)) {
            throw new FraudException("Anomaly detected");
        }
        
        // Identity verification (Article 17)
        if (!authService.verifyIdentity(req.getSsn())) {
            throw new AuthException("Identity verification failed");
        }
        
        return loanService.approve(req);
    }
}

Regulatory Change Response

When regulations change:

1. Update training data
2. Retrain LoRA (2-3 days)
3. Auto-scan existing code → Detect violations

---

Scenario 4: Multi-Customer Operations

When an SI company operates multiple customers on the same platform, per-customer LoRA adapters are hot-swapped. The GLM-5-32B in the table below is not a specific public model but an example name standing for a self-hosted base model, so replace it with the base model you actually use.

Adapter routing is only the mechanism that selects a different LoRA weight set per customer; by itself it is not a tenant authorization or isolation boundary. Data and request isolation between customers must be guaranteed by separate controls such as authentication/authorization and namespace/network separation.

Per-Customer Configuration

Table 4. Per-customer model, LoRA, and RAG configuration for multi-customer operations

Customer	Domain	Base Model	LoRA	RAG
Bank A	Core banking	GLM-5-32B (example)	Bank-Core	Bank-API
Securities B	Order execution	GLM-5-32B (example)	Securities-Order	Securities-API
Insurance C	Contract management	GLM-5-32B (example)	Insurance-Contract	Insurance-API

For Multi-LoRA deployment and per-customer routing implementation, see Custom Model Pipeline — LoRA Training & Deployment Pipeline.

---

Evaluation Pipeline

Continuously validate the quality of domain-specialized models. Follow these evaluation methods and baselines:

• RAGAS Evaluation Framework: Measures RAG accuracy (faithfulness, relevancy, context recall)
• Custom Model Pipeline — Evaluation Pipeline: LoRA adapter evaluation metrics, A/B testing

---

Phase-by-Phase Adoption Roadmap

The durations and costs in the table below are planning assumptions (illustrative figures), not measured values. "70% savings" is an illustrative target that assumes SLM Cascade hands part of the requests to small models; "+$500/month" assumes only the additional monthly cost of the SLM serving infrastructure, and "GPU $2K" assumes only the GPU usage cost for LoRA training, excluding labor, data preparation, and evaluation costs. Re-estimate the actual durations and costs by measuring with the target environment's traffic, models, and instance prices.

Table 5. Duration, configuration, effect, and cost by adoption phase

Phase	Duration	Configuration	Effect	Cost
1	Immediate	Steering + Playbook	Compliance + Basic quality	Free
2	1-2 weeks	+ VectorRAG (Milvus)	Internal knowledge accuracy improvement	Infrastructure
3	2-4 weeks	+ SLM Cascade	Cost optimization (70% savings)	+$500/month
4	1-2 months	+ LoRA Fine-tuning	Domain expertise + Style consistency	GPU $2K

For detailed per-phase implementation guides, see the Custom Model Pipeline Guide.

---

References

Official Documentation

• LoRA Paper (Hu et al., 2021)
• QLoRA Paper (Dettmers et al., 2023)
• vLLM Multi-LoRA
• LangChain Retrieval and RAG
• Neo4j GraphRAG
• RAGAS Evaluation
• Unsloth Fast Training
• NeMo Framework

Related Documentation

• Custom Model Pipeline
• RAGAS Evaluation Framework