Accidentally making a duplicate copy of one of the .cs files in the App_Code folder will, yes, sabotage a web site.

C#'s compiler will not tolerate two classes with both the same name and same namespace unless the partial keyword is being used which will not apply in this circumstance. As all files in the root folder of a solutionless ASP.NET web site are included in the web site, in the absence of a .csproj file decrying what to include, a duplicate of any one file containing a class should do a pretty good job of wrecking things.