I left my security system switched off on purpose
Here is a decision that sounds daft until you have seen the data. I built a system that can block unauthorised changes to my live servers, and then I deliberately left it switched off. Not broken. Off, on purpose, running in a mode where it watches and records but stops nothing.
The instinct is to turn the lock
And the trap in doing it too early.
The instinct in security is always to turn the lock. Build the wall, enforce the rule, slam the door. It feels like the responsible thing. But there is a trap in it, and the only way to spot it is to look before you leap.
A system that starts enforcing rules on day one does not actually know what your normal looks like yet. It only knows the rules you dreamed up at your desk. And real, everyday work is always messier and more varied than the tidy picture in your head. So the first thing a keen new gatekeeper does is block you, the legitimate operator, getting on with your ordinary job, because your ordinary job does not perfectly match rules written before you had watched a single real day.
So I watched instead
Block nothing, record everything.
So I ran mine in watch mode instead. For weeks it saw everything, blocked nothing, and quietly wrote down every action it would have stopped if it had been enforcing. Then I read the log back.
What the log actually showed
Thousands of entries, nearly all of them me.
Thousands of entries. Nearly every one was me, doing normal, legitimate work. Routine housekeeping, ordinary updates, the everyday running of the place. The number of actual intruders trying to do actual harm was, near enough, none.
So if I had switched that lock to enforcing on day one, the main thing it would have done is get in my own way, thousands of times over, while stopping almost nothing worth stopping. It would have felt secure and mostly just been a nuisance. Worse, after the fiftieth false alarm I would have started ignoring it, which is exactly how a real alert slips through.
Watch before you tighten
Learn the real normal, then aim.
It is not really a lesson about servers. Before you enforce any rule, on people or on machines, watch what actually happens for a while first. Learn the real normal, not the one you imagined, and then you can tighten deliberately, aimed at the genuine problems, instead of blindly, aimed at everyone.
Mine is still in watch mode. The log says switching it on today would mostly block routine work rather than any threat, so it stays watching until the evidence says different. Watched, recorded and reversible is an honest place to sit, and a lot more use than a lock that mostly locks you out.
For the techies
The short version of what actually sits under this.
This is how we build
Careful, evidence-led, and honest about what is actually protecting you. If that is your kind of thing, get in touch.
Drop me a line