The New Reality of AI-Assisted Development

Modern developers increasingly rely on AI coding assistants to generate code and suggest dependencies.

Typical workflow:

Developer Prompt
      │
      
AI Generates Code
      │
      
Suggested Dependency Installation
      │
      
Developer Executes pip / npm install

The problem is that AI does not validate security of the dependencies it suggests.

This creates a new attack surface in the software supply chain.

A supply-chain attack occurs when attackers compromise a trusted third-party component such as a dependency or library to inject malicious code into downstream software.

Where the Attack Happens

AI Generated Code
        │
        
Dependency Suggestion
        │
        
Package Registry (PyPI / NPM)
        │
        
Malicious Package Installed
        │
        
Post-Install Script Executes
        │
        
Data Exfiltration / Backdoor

Attackers exploit the trust developers place in open-source libraries, inserting malicious code that executes once installed.

The Most Common Attack Technique: Typosquatting

Example visualization:

Legitimate Package:
requests

Malicious Package:
requestss
reqeusts
requsets

Attackers publish packages with names very similar to legitimate libraries hoping developers install them accidentally.

Once installed, the malicious package may:

• steal environment variables
  extract SSH keys

• capture cloud credentials

• download remote malware

Real-World Supply Chain Incidents

Compromised AI-related Python package

A compromised version of the AI library LiteLLM included a backdoor capable of stealing sensitive data such as SSH keys, Kubernetes secrets, and cloud credentials.

Malicious dependency injection

Attackers compromised versions of a popular JavaScript library and inserted a dependency that installed a remote access trojan during installation.

Malicious packages in public registries

Security researchers regularly discover packages in repositories like PyPI and npm that execute malware during installation or steal developer data.

What Happens After Installation

Typical malicious package behavior:

pip install malicious_package
        │
        
setup.py or install script runs
        │
        
Collect system information
        │
        
Send credentials to attacker server
        │
        
Download secondary malware

Many malicious packages hide code in setup scripts that run automatically during installation.

Why AI Makes This Risk Worse

AI assistants can:

• hallucinate non-existent libraries

• suggest outdated dependencies

• recommend vulnerable packages

  Some studies show LLMs may generate fictitious dependencies that attackers could later publish, creating a new attack vector.

Secure Workflow for AI-Generated Code

Recommended developer workflow:

AI Generated Code
        │
        
Verify Dependency Source
        │
        
Run Vulnerability Scan
(pip-audit / npm audit)
        │
        
Static Code Security Scan
(Semgrep / SAST)
        │
        
Secret Detection
(Gitleaks)
        │
        
Run Code in Sandbox
(Docker / isolated environment)

This significantly reduces the risk of supply chain compromise.

The Key Insight

AI tools accelerate development.

But they also introduce automated trust in external dependencies.

And attackers exploit exactly that trust.

The most dangerous line of code today might simply be:

pip install <unknown package>

Takeaway for AI Engineers

When using AI-generated code:

• Never blindly trust dependencies

• Validate libraries before installation

• Implement security scanning in CI/CD

• Run AI-generated code in sandbox environments

AI is transforming software development.

But secure development practices must evolve with it.

Typical workflow:

Developer Prompt
      │
AI Generates Code
      │
Suggested Dependency Installation
      │
Developer Executes pip / npm install

The problem is that AI does not validate security of the dependencies it suggests.

This creates a new attack surface in the software supply chain.

A supply-chain attack occurs when attackers compromise a trusted third-party component such as a dependency or library to inject malicious code into downstream software.

Where the Attack Happens

AI Generated Code
        │
        
Dependency Suggestion
        │
        
Package Registry (PyPI / NPM)
        │
        
Malicious Package Installed
        │
        
Post-Install Script Executes
        │
        
Data Exfiltration / Backdoor

Attackers exploit the trust developers place in open-source libraries, inserting malicious code that executes once installed.

The Most Common Attack Technique: Typosquatting

Example visualization:

Legitimate Package:
requests

Malicious Package:
requestss
reqeusts
requsets

Attackers publish packages with names very similar to legitimate libraries hoping developers install them accidentally.

Once installed, the malicious package may:

• steal environment variables

• extract SSH keys

• capture cloud credentials

• download remote malware

Real-World Supply Chain Incidents

Compromised AI-related Python package

A compromised version of the AI library LiteLLM included a backdoor capable of stealing sensitive data such as SSH keys, Kubernetes secrets, and cloud credentials.

Malicious dependency injection

Attackers compromised versions of a popular JavaScript library and inserted a dependency that installed a remote access trojan during installation.

Malicious packages in public registries

Security researchers regularly discover packages in repositories like PyPI and npm that execute malware during installation or steal developer data.

Lets deep dive after installation

Typical malicious package behavior:

pip install malicious_package
        │
        
setup.py or install script runs
        │
        
Collect system information
        │
        
Send credentials to attacker server
        │
        
Download secondary malware

Many malicious packages hide code in setup scripts that run automatically during installation.

Why AI Makes This Risk Worse

AI assistants can:

• hallucinate non-existent libraries

• suggest outdated dependencies

• recommend vulnerable packages

Some studies show LLMs may generate fictitious dependencies that attackers could later publish, creating a new attack vector.

Secure Workflow for AI-Generated Code

Recommended developer workflow:

AI Generated Code
        │
        
Verify Dependency Source
        │
        
Run Vulnerability Scan
(pip-audit / npm audit)
        │
        
Static Code Security Scan
(Semgrep / SAST)
        │
        
Secret Detection
(Gitleaks)
        │
        
Run Code in Sandbox
(Docker / isolated environment)

This significantly reduces the risk of supply chain compromise.

Key takeaway for us

AI tools accelerate development.

But they also introduce automated trust in external dependencies.

And attackers exploit exactly that trust.

The most dangerous line of code today might simply be:

pip install <unknown package>

When using AI-generated code:

• Never blindly trust dependencies

• Validate libraries before installation

• Implement security scanning in CI/CD

• Run AI-generated code in sandbox environments

AI is transforming software development.

But secure development practices must evolve with it.

Why Benchmarking is Important?

Computer scientist discovered various way to measure intelligence, capacity, capability, using various checks for LLM model and standardized them.

you will have seen some models when given update like gemini, grok, Qwen, Deepseek, Sarvam etc.
below is one snippet which is released recently.

Below is the benchmark report of the Gemini 3.1 Pro Preview released by Google yesterday

Now as you can see their multiple Benchmarks each one has specific things which validates.

You can also see for one of popular opensource model developed by qwen

Now you would have observed there are multiple ways of benchmarking, and every model have to go through these tests.

So, a question arises here why you should you be bothered about these bench marking methodologies.?

Now let's take analogy of buying any product, you should have purchased any items for instance take example of buying a laptop what you do.

You research and take note of

1. you gather what you want (like what work you want to do example gaming, video-editing)

2. you check what are the criteria that laptop should have like higher Ram, must have external Graphics card, higher CPU model other aspects too.

So, idea behind this analogy is to see what the things are required for your requirement it gives you a all idea about choosing a product which suits you.

That is why it is required to understand the benchmarking and model attributes, that will help you to choose models for your specific work or requirement when you build any product.

Now wrapping it up for now dropping you a question here

As you know these benchmarks are standardized then any model trainer would train them to pass those particular benchmarks then how the credibility of the model is met in large context?

Leaving you with that.
Comment your thoughts on that.
In the upcoming blogs we will go through each popular benchmarking models one by one with their evolution

Subscribe on youtube

Tune into podcast

A vector database is a specialized storage system designed to store, index, and search vector embeddings.

**How it works?

1. Vectorization:**

Raw data (text, image, audio) is processed by ml model to create a digital fingerprint (vector embedding) a large list of numbers which represents the item’s features.

2. Indexing

Organizes these vectors in multi-dimensional space. Related items are positioned mathematically closer to each other

3. Similarity Search

When a query is made, search is converted to vector and then database identifies nearest neighbours

Flow depicting the use of vector embedding

Why Vector Database?

Question comes in mind when there are already so many databases like (MySQL, Oracle, NoSQL, influx) available why need of vector database.

The selection and use of these databases depend on the data structure which we want to store in them.

As we know the data which we want to store are unstructured like text, images, audio. We required create or use database which can handle high-dimensional vector embeddings.
other aspects which lead to use is vector database allows semantic search or similarity search.

Majorly used vector databases by popular AI Models

What’s next … which vector database to choose and comparison of popular vector databases