How to stop a zombie IIS application pool

Have you ever stopped an IIS application pool, just to find it running later? This a quick post to explain why this happens, and how to correctly stop the app pool so it stays down.

(check our Reset, Restart and Recycle IIS guide for more on effectively restarting your application pools and avoiding the problematic side effects).

Why stop an application pool in the first place?

You may choose to stop one of your app pools if:

  • It’s having an excessive impact on your server.
  • It is hosting applications that you don’t want running (unused, broken, or for security reasons).
  • It’s a copy of another application that’s already running (and you don’t want it conflicting or running twice).
  • You are doing a performance test on another app, and you want everything else stopped.
  • You are updating the applications and you don’t want them restarting during the update.

Stopping apppools for security and performance reasons can actually be an effective strategy when dealing with many customers/apps on the server, or removing unused applications that can become a DOS target.

Why does the stopped application pool come back to life?

The reason for this is simple. The Windows Process Activation Service (WAS, WPAS) will start all application pools when it starts.

So, if WAS restarts for any reason, it will re-start your stopped application pools. 

This will happen in all of these cases:

  • IISRESET
  • Net stop was /yes && net start w3svc (restarting IIS services)
  • Rebooting your server VM

When someone makes a request to the application that’s mapped to your application pool, it will then start a worker process.

The application pool will also start a worker process when using the AlwaysRunning mode. Which we totally recommend for your important production sites, as you can see in our Maximize IIS Application pool availability guide.

How do I stop the pool so it stays stopped?

Easy.

You must also configure the application pool to set autoStart=false. This way, WAS knows not to start it unless you explicitly choose to start it.

Here is the compact command:

%windir%\system32\inetsrv\appcmd set apppool POOLNAME /autoStart:false
%windir%\system32\inetsrv\appcmd stop apppool POOLNAME 

This causes this applicationHost.config configuration to be written in your application pool’s definition:

<add name="TestApp" autoStart="false" ...>

To start the application pool back up

If and when you want to get the pool started back up:

%windir%\system32\inetsrv\appcmd set apppool POOLNAME /autoStart:true
%windir%\system32\inetsrv\appcmd start apppool POOLNAME 

You can skip the autoStart:true command if you want to continue manually managing the pool’s uptime, and just start/stop it when needed.

That’s it. No more zombie application pools.

P.S. If you haven’t already, check out our Reset, Restart, Recycle IIS guide for useful tips on how to correctly restart production apppools with minimum downtime.

Guide: IIS Thread pool optimization

We’ve just released our new IIS Threading guide, which explains how the IIS thread pool works and how it affects your website performance.

It turns out that many people don’t have a clear sense of how the IIS thread pool relates to hangs and slowdowns in their website.

So, we took the opportunity to clear things up, and explain how IIS thread pool exhaustion or thread starvation actually affects IIS site performance.

This includes:

  1. 503 Queue Full errors, or application pool queue buildup
  2. Reduced RPS, especially for high performance websites

We also explain how most traditional hangs and slow loads are NOT caused by IIS threadpool issues, but instead the application-specific thread pools like the CLR thread pool, or the Classic ASP thread pool.

Monitoring IIS thread performance

We also show a simple way to monitor IIS counters to detect IIS pool issues.

If you have LeanSentry installed on the server, you can get a lot deeper into actually diagnosing most thread pool issues down to code (part of our application pool failure diagnostics). We’ll cover this well.

LeanSentry diagnosing IIS thread pool issues causing 503 queue full errors
LeanSentry diagnosing an 503 queue full incident down to code causing CPU overload, which is then causing IIS thread delays.

Optimizing the thread pool

In the guide, we dig into the specific causes of threading issues, that actually happen. This is based on monitoring 30K+ IIS websites with LeanSentry in the last decade.

Surprisingly, there are only a couple causes of thread pool issues that matter.

We’ll cover each one, and how to configure the IIS threadpool to resolve them.

For the juicy details, head over to the thread pool guide.

Guide: Maximum IIS application pool availability

Building on our last week’s Reset, Restart, Recycle IIS guide, this week we are digging deeper into improving IIS application pool availability.

Head over to How to maximize IIS application pool availability to see how you can tweak your IIS websites to:

  1. Start in 100%-warm state, for best cold-start performance (even after a production restart or recycle)
  2. Have zero startup delay (despite the warmup we just mentioned)

Yes, basically you CAN have your cake and eat it too.

Meaning your website is always warm, and can recycle without the users feeling it. This requires getting a lot of pieces right, and guide shows you exactly how to do that.

Best of all, it requires ZERO code.

Configure it all instantly with the ConfigureWarmup tool

We have a tool that will automatically configure the warmup configuration for your website. We’ll be releasing this tool shortly, but if you want it now, be sure to sign up for the Guide newsletter early access. This will allow you to get all our guides, and tools, before they are publicly released online.

Guide: how to restart IIS websites correctly

Our first guide in the IIS Expert guides series is out.

This one is about the right ways to restart IIS websites, and the production impacts of the different ways customers do it.

We also discuss in detail WHEN you should restart IIS or recycle your application pool, and when you shouldn’t … and the better alternatives to restarting.

Improving IIS application pool availability

This post is part 1 of our upcoming series on how to configure IIS for maximum availability. The next guide will touch on how to configure your sites for 100% warm operation, with zero startup delays. We also have a tool called ConfigureWarmup that can do this for you automatically, and can even test your warmup performance. Stay tuned for these coming in the next two weeks.

P.S. Be sure to sign up for the IIS guides newsletter to get early access to all of these.

Expert IIS guides series

It’s hard to believe that LeanSentry is turning 10!

During the last 10 years, we went from a collection of IIS troubleshooting tools that Mike wrote to help fix performance problems for large IIS sites like MySpace, Microsoft mobile, and Vevo to a comprehensive diagnostic service for troubleshooting and optimizing IIS websites.

What’s more, we’ve worked with thousands of companies and IT/developer pros like you to help solve real issues for real applications.

During this time, the number one challenge we’ve heard from our customers is the need for best-practice knowledge, and real-world skills on how to do troubleshooting and optimization. Even with a diagnostic service like LeanSentry, many customers still found it difficult to make the actual changes they needed to resolve their problems.

Get the guides

We personally worked with, and helped thousands of you to do this. Now, we’ll be sharing our techniques and tools we’ve used through the years to empower as many people as possible to run reliable, high performance websites on the Microsoft web platform.

We’ll cover topics like:

  1. How to configure your IIS website for maximum availability,
  2. How to save cloud hosting costs with smart optimizations,
  3. What kind of IIS monitoring helps you fix website issues, and what is a waste of your time,
  4. How to quickly recover from memory leaks, hangs, high CPU issues, queueing, and others,
  5. Best practice application patterns ASP.NET developers can use to build faster, more reliable, more scalable apps,
  6. Tools we use to all these things correctly, and much more

We’ll also look at a bunch of things that DON’T WORK. There are a lot of those … including popular things that take down your website, break your webserver, or cause you to spend 4x more money on your cloud hosting.

To be the first to receive content, be sure to head over to https://www.leansentry.com/guide and sign up for the Guide newsletter.

Sign up for the expert IIS guides newsletter.

See you soon!

LeanSentry How-To: IIS & ASP.NET recommendations from our team

In the last 2 years, we’ve helped thousands of big and small companies to improve their IIS servers & ASP.NET applications.

Today, we are announcing LeanSentry How-To: a series of best-practice guides to help you solve common performance problems & improve your site health.

We’ll cover topics like:

1. Azure best practices
2. IIS & ASP.NET performance
3. Improving application & server stability
4. Advanced tools & techniques for troubleshooting production problems

To get these, sign up for our How-To series at LeanSentry How-To.

If you have specific areas or questions you would like to see covered, email us or post them here anytime.

Fix IIS & ASP.NET hangs faster with Hang diagnostics

Automatic hang diagnostics for IIS & ASP.NET apps has been one of LeanSentry’s most popular features.

Now, Hang diagnostics are getting even better, with more expert insight into hangs, better root cause detection, and more code-level data for your developers.

Automatically diagnose ASP.NET hangs

LeanSentry’s Hang diagnostics feature helps you resolve your website’s slowdowns, better than you can with generic monitoring tools. It does this by detecting and by automatically diagnosing dozens of common IIS, ASP.NET, and Classic ASP performance problems whenever your site experiences them.

No intrusive profilers, DebugDiag, or other tools needed. Like most of LeanSentry, hang diagnostics have virtually zero overhead during normal operation, and only a small overhead (5-10 seconds of analysis) when a problem is confirmed. This means you can add LeanSentry’s deep diagnostic insight to your existing monitoring without any conflicts or performance drops.

What’s new?

A lot! We’ve improved the diagnostic algorithms to detect more problems. Then there is the brand new diagnostic report interface that gives you better guidance, and more information on what caused the hangs.

See the actual IIS & ASP.NET operation

See the real request flow during ASP.NET hangs
See the actual request flow, and the key IIS & ASP.NET performance issues during the hang.

Expert guidance

Full ASP.NET hang analysis
LeanSentry analyzes the specific causes of the hang in your app, and gives you best practice guidance on how to address it.

Identify the function, slow SQL, REST call causing blocking

Diagnose ASP.NET hangs down to code, SQL query, and more
LeanSentry automatically determines the application code causing request blocking or thread pool exhaustion, and shows your the line of code, SQL query, REST url, or file being accessed.

Try it out

If you are still troubleshooting hangs the hard way, or trying to use the generic transaction monitoring tools, you are missing out. To learn more, check out LeanSentry’s hang diagnostics, and do a trial to see it for yourself.

LeanSentry is back, and better than ever!

You may have noticed that we’ve been very quiet in the last year.  The reason: as a small team, we found the initial demand for LeanSentry a bit overwhelming. This made it difficult to focus on building the kind of product we envisioned.

So, we decided to take a year to innovate our product, and deliver the kind of experience that our customers wanted.

Today, we are back with a brand new www.leansentry.com and whole new generation of our service.

What does LeanSentry do?

LeanSentry uses expert analysis techniques, many developed at Microsoft, to give you deeper insight into your web apps.

We automatically analyze IIS and ASP.NET performance, troubleshoot website hangs, tune .NET CPU and memory usage, troubleshoot errors, and much more.

The best part? It’s lightweight, cannot affect your applications, and can run alongside any existing APM or monitoring tool.

What’s different about LeanSentry?

In short, everything!

We have a brand new, real-time interface that makes it easy to access information with just a few clicks.

Our diagnostics have been upgraded to be way easier, and give you lot a lot more insight into your stack’s performance.

LeanSentry Dashboard
We have a simpler, faster deployment that lets you add LeanSentry to your production servers, Windows Azure or Amazon EC2 in less than 5 minutes, with no restarts or iisreset.

Here are some highlights for a quick glance:

Give me!

If you’ve tried LeanSentry a while back, you simply must see it again.

If you have not, this is a good time to check it out! Head over to www.leansentry.com and sign up for the free trial to see for yourself.

To all our customers and everyone that supported us so far, thank you! We hope the new service makes your life that much better. Stay tuned for a lot more about the new product, our story, and much more in the coming weeks!

New Performance page helps you track website CPU/Memory & slow requests

This week, we have another big improvement for you.

We completely remade our Performance page, and made it a lot easier to view your website performance data.

What’s there?

  • Slow requests tab shows you what operations caused slow requests to your site.
  • Resource use tab shows you the CPU, Memory, and Network usage by the website … compare it across all servers … and give you the diagnostic data to explain what caused the usage.

As a bonus, you can enable CPU profiling, Memory diagnostics, and slow operation tracking directly from the page.

Above all, the new interface is clean and wonderfully simple.

This is available immediately in your LeanSentry account! Just log in and go to the Performance page for any of your websites.

Don’t have an account? Get a trial account to see it for yourself.

We have more great stuff coming next week, so stay tuned!

Fix IIS hangs – troubleshooting guide!

IIS hangs troubleshooting guide

We’ve been working hard on LeanSentry’s automatic diagnostics, to make sure it can detect and diagnose most production issues for you.

If you are not using LeanSentry yet, that means you are still doing troubleshooting by hand. To help, Mike recently created the LeanSentry Production troubleshooting course, an email course based with his own debugging techniques.

Today, we published an online guide for the course’s most popular troubleshooting topic, how to troubleshoot IIS website hangs.

Fix IIS hangs

The guide gives practical tips on how to isolate, and diagnose the hang so you can fix it. All you’ll need is the standard Microsoft tools and some time.

Here is a sneak peak:

Fix IIS hangs - table of contents

Want more IIS & ASP.NET troubleshooting guides?

If we get good feedback on the guide, we plan to turn the rest of the class into online guides for everyone to use.

It helps us raise our profile, and its an awesome way for us to contribute to the IIS & ASP.NET community.

So, go check out the “Fix IIS hangs” guide, share it, and leave your feedback on Mike’s blog!

Best,

The LeanSentry team