make Web.config look into another file for appSettings

<appSettings configSource="config\whatever.config" />

...will defer to another config file. In the other file you should have the open and close tags for appSettings and a bunch of keys between them. Do a variant of the same thing for connectionStrings.