When considering layer performance, our usual goal is to have a layer render in under 2 seconds, and simple layers can often render in under 1 second. When setting up an MXD in ArcMap, the user can get a rough idea of how long layers take to render by turning them on and off to see how long they take to render on the desktop. Any complex layers (parcels, local streets, detailed coastlines) should be limited to a closer zoom, and any general layers (county boundary, highways, large polygons) can be available at full extent.
If you are using ArcGIS 9.3.1, there is a performance analysis tool available on the “Map Service Publishing Toolbar”. Right-click the main menu in ArcMap to access the Map Service Publishing toolbar. This is helpful for overall map service performance, and can also give feedback on layers that should be limited to a larger scale. More info and instructions on how to use the Map Service Publishing Toolbar from ESRI here.
If you are using ArcGIS 9.2 or 9.3, there is a great script available called MXDPERFSTAT. The script is a free download from ESRI ArcScripts and it works with 9.2, 9.3, and 9.3.1.
MXDPERFSTAT is fairly straightforward to run, you launch it from a command prompt and it will give you feedback on your MXD performance for ArcGIS Server. It analyses your MXD at different scales and locations, which you can set yourself or accept the defaults. It goes through and turns on each layer of your MXD one by one and assesses it for performance. When complete, it outputs an XML file which shows the refresh time in seconds at each scale for the whole map, and then more detailed information for each layer at each scale, including warnings, recommendations to improve performance, number of features and vertices drawn, and layer spatial reference, to name a few. I found it took approximately 15 minutes to run the script on a site with 50 layers. The more complex layers (parcels, streets) took noticeably longer to process than the simple layers.