Actually google values backlinks from .edu websites. don't know about .gov domains. Even if you build backlinks from .edu domains make sure they are relevant cause google cares more about relevance than they care about domains.
You should not make all of your backlinks from .edu and.gov sites because its not natural and its possible that you loss your ranking on Google. All seo experts say control the number of .edu and .gov backlinks.
Try to get backlink from different websites, it's more valuable than just get them from one or two. Of course, if the .edu and .gov websites got very high PageRank and your backlinks are do follow, it's worthy!
Google analyze the "naturalness" of links, they've obviously seen a lot of "good" and "spammy" sites so if the distribution of links to your site doesn't look natural then there's a good chance google might flag your site as spammy.