Quick description
Many groups are defined indirectly, for instance by writing down a list of generators and relations between those generators. As such, it is not always obvious whether a given group element is trivial (equal to the identity) or not. (Indeed, the word problem for groups is undecidable in general.) But in many cases, one can demonstrate the nontriviality of a group element by first constructing a homomorphism from to a much better understood group , and showing that the image of is nontrivial. An important special subcase of this strategy is to produce an action of on some set , and show that the action of on is nontrivial, for instance by producing an element of such that .
Prerequisites
Elementary group theory
General discussion
We'll start by giving three different approaches to the same problem.
Problem: In the group generated by two generators subject to the relation , show that and are both nontrivial.
The first solution to this is based on an easy observation.
Example 1
The group is the freest possible group in which satisfy the given relation. So if we can find any other group containing nontrivial elements that satisfy the relation then we will be done. In this case, after a small amount of thought we might think of the symmetry group of the triangle, also known as , which is generated by a rotation of order and a reflection of order . The reflection conjugates the rotation to its inverse, so . It is easy to deduce that , so there is a homomorphism from to that sends to and to . As and are both nontrivial, it follows that and are too. [Another example would be the affine transformations and .]
Next, we'll give a different approach to this problem.
Example 2
For , the problem is quite easy, as one can take advantage of the fact that the only relation in the group preserves the number of 's on both sides. To formalize this, we define the homomorphism that maps to and to ; such a homomorphism exists because it respects the relations defining the group (here we are using the universality properties of a group defined by a set of relations). Since is nontrivial, is nontrivial.
The situation is more delicate for , and here we will use a more ad hoc argument. We first observe that the relation implies a number of variants: , , and . Thus, given any word involving and , one can move all the symbols to the left (using the identities , ) and all the symbols to the right (using the identities and ). Thus every word can be written in the form for some nonnegative and integers . But this is not quite the end of the story, as we can also simplify the word using the relation for any , which implies that for any . So what we will do here is define a set to be the set of all formal word strings for integer and nonnegative , modulo the equivalence relation generated by the relations for integer and nonnegative . We can then define an action of the generators of on by the formal operation of adjoining the generator to the left of and then simplifying using the relations available. More concretely:

The action of on is .

The action of on is if , or if .

The action of on is .

The action of on is .
One can then check that these actions are indeed actions on (i.e. they respect the equivalence relation) and that the action is compatible with the relation , and so it extends to an action of on . The action of on is nontrivial (one can check, for instance, that and are inequivalent in ) and so must be nontrivial in . (In fact, with a bit more work one can show that acts in a transitive, faithful, and free manner on , and so can in fact be placed in onetoone correspondence with .)
The technique used in Example 2 is actually an instance of a much more general theory.
Example 3
The group is easily seen to be an HNN extension of : the stable letter conjugates to . Britton's Lemma asserts that if a product of the generators
is equal to the identity then the expression for is reducible, in the sense that it contains a subword of the form or for some integer . In either case, the relation of can be used to shorten . In particular, it follows that and are nontrivial.
The proof of Britton's Lemma is a direct generalization of the argument of Example 2—one lets be the set of all irreducible strings, and proves that the natural action of on is free and transitive.
For a more geometric understanding of the action of on , see To prove a theorem about graphs of groups translate the problem into a question about group actions on trees.
Comments
Inline comments
The following comments were made inline in the article. You can click on 'view commented text' to see precisely where they were made.
Well, there's a theory for
Thu, 30/04/2009  20:51 — wiltonWell, there's a theory for dealing with this. When you use it, then things are certainly simpler!
The group you're considering here is the Baumslag–Solitar group BS(1,2). The standard way to prove that is nontrivial is to observe that the group is an HNN extension of and appeal to Britton's Lemma. You're basically reproving Britton's Lemma in this case.
Another (morally equivalent) point of view is to say that the object on which your group acts is the universal cover of a space constructed by gluing one end of a cylinder to itself via a 2to1 covering map. The fact that the circle in the cylinder lifts to a line in the universal cover shows has infinite order.
I'd be tempted to do an example like the (2,3,7)triangle group, and construct the action on the hyperbolic plane.
Perhaps we can segue this article into a discussion of Britton's Lemma, graphs of groups etc. Hmmm.
Isn't the relation satisfied
Thu, 30/04/2009  21:03 — gowersIsn't the relation satisfied in the dihedral group ? In other words, if we add the relations , we get a group where we know that and are nontrivial (e.g. because it acts faithfully on a triangle). So there's our homomorphism. Or am I missing something?
Ahh, that's much easier :)
Thu, 30/04/2009  23:02 — taoMore generally, I guess one can go look for nontrivial matrices which are conjugate to their square (not hard to guess such matrices, after thinking a little about what their eigenvalues must be like), and that gives a whole bunch of useful representations. Easy in hindsight :) [and, annoyingly enough, I actually knew this; this is how one proves the Frobenius lemma that the irreducible representations of are large. I should have known better than to try to attack the word problem directly...]
I'm not sure how to structure the article now; I feel that my pedestrian way of doing things is instructive, if perhaps only in the negative sense of showing what not to do. Perhaps we can give several different proofs of the same result?
I didn't know about Britton's lemma, so I learned something today!
And, it injects into the ax+b group
Thu, 30/04/2009  23:11 — taoOne can map a into the translation x > x+1, and b to the dilation x > x/2, and it's looking like this is a faithful representation. Well, it's still a good example of the general principle, at least...
Yes. BS(1,2) is isomorphic
Thu, 30/04/2009  23:19 — wiltonYes. BS(1,2) is isomorphic to the semidirect product of Q_2 and Z, where Z acts via multiplication by 2. Which gives yet another way to see do it, by considering actions on pairs of these things.
I rather like the idea of including all these different approaches. There are interesting stories to be told about which generalize to other, similar, groups. For instance, Tim's nice observation works because BS(1,2) is residually finite. But BS(2,3) isn't, so it won't work there.
One other thing is that in an
Fri, 01/05/2009  06:33 — gowersOne other thing is that in an article I wrote on group presentations I gave two proofs that the group with two generators and and the relations is infinite. One was a direct attack and the other was to find an action on with an infinite orbit. It's similar enough that some kind of crosslinking might be good (or alternatively one might just repeat the example here – my view is that it's fine if the same example appears in more than one article). I won't do anything yet, given that the structure of this article is not yet clear.