That is because the vertices in your vertex group belong to more than one edge or face, so when you select them from vertex group, you select all edges/faces that are formed from the assigned vertices in the vertex group.
One way around this would be to have Edge Groups and Face Groups, but we don’t have that yet. I don’t know a way to do this in Blender, but I will look into it, bear in mind it is Christmas though.
Blender does only support vertex assign for groups, its very deeply rooted into the code/architecture of blender, reason why it has not been rewritten until now as it would require a large rewrite of the blender code.
If we ever get edge/face group support it would only happen when the large code rewrite happens.
Since blender 3.0 is on the horizon i wonder if it can be changed, since that version might break a lot of stuff codewise, so a break in compatibility might be acceptable.
That missing part is a fairly big one, though, right? Edge marks (bevels, creases, seams, sharps) are currently drawn as a part of the edges themselves, with just a width and color change, and not as an overlay. I can’t really see a way to visualize arbitrary attributes using that approach, which means there’s a whole overlay system (in addition to the actual workflow stuff) that needs to be hashed out before edge groups can be added to the Soon™ list.