On the Isomorphism Problem of Coxeter Groups and Related Topics

Koji Nuida

Introduction

A Coxeter group is a group which admits a group presentation of a certain typical type. One of the most active fields in the recent researches on Coxeter groups is the isomorphism problem of Coxeter groups; that is the problem of deciding which Coxeter groups are isomorphic as abstract groups, and of studying further properties of isomorphisms between Coxeter groups. Several results and observations for this problem have been given in this decade, particularly in the case of finitely-generated Coxeter groups. However, only a few observations have been given for non-finitely-generated cases. The aim of this dissertation is to give a breakthrough for the isomorphism problem of Coxeter groups without the assumption of the finiteness of generators. Moreover, we also give some more results on abstract groups and on Coxeter groups, which are of independent importance and interest.

This dissertation consists of two parts, each containing an independent paper discussing topics related to the isomorphism problem of Coxeter groups. In Part I, entitled “On the direct indecomposability of infinite irreducible Coxeter groups and the Isomorphism Problem of Coxeter groups”, we study a relationship between the isomorphism problem and irreducible components of Coxeter groups; our result reduces the problem to the case of infinite irreducible Coxeter groups. Moreover, an analogue of the Krull-Remak-Schmidt Theorem on indecomposable decompositions of abstract groups is provided, and the automorphism groups of Coxeter groups and the centralizers of normal subgroups in Coxeter groups generated by involutions are also described.

In Part II, entitled “Almost central involutions in split extensions of Coxeter groups by graph automorphisms”, we give a sufficient condition for an isomorphism between two Coxeter groups to be reflection-preserving. This is an important step of the study of the isomorphism problem of Coxeter groups. More detailed examination of this condition will be done in a forthcoming paper of the author which is now in preparation. We also study a split extension of any Coxeter group by Coxeter graph automorphisms, determining the involutions in such an extension whose centralizer has finite index. Moreover, certain special elements of Coxeter groups and the fixed-point subgroups in Coxeter groups by Coxeter graph automorphisms are also studied.

Acknowledgement. The author would like to express his deep gratitude to everyone who helped him. Especially, his supervisor Professor Itaru Terada gave suggestions toward improvement of his works and encouragement. Professor Kazuhiko Koike also gave several helpful comments and encouragement at the seminar. A motivation for this research originated from a suggestive comment of Doctor Manabu Hagiwara. Professor Bernhard Mühlherr kindly sent his illuminating survey on the isomorphism problem to the author and gave some comments. The JSPS Research Fellowship gave financial support throughout this research, and the Institut Mittag-Leffler kindly invited the author to a research program in spring 2005. Finally, the author thanks his wife, Eriko, very much for her warm supports and hopeful encouragement at all times.

Part I

On the direct indecomposability of infinite irreducible Coxeter groups

and the Isomorphism Problem of Coxeter groups

ON THE DIRECT INDECOMPOSABILITY OF INFINITE IRREDUCIBLE COXETER GROUPS AND THE ISOMORPHISM

PROBLEM OF COXETER GROUPS

KOJI NUIDA

In this paper we prove that any irreducible Coxeter group of infinite order, which is possibly of infinite rank, is directly indecomposable as an abstract group. The key ingredient of the proof is that we can determine, for an irreducible Coxeter group W, the centralizers in W of the normal subgroups of W that are generated by involutions. As a consequence, the problem of deciding whether two general Coxeter groups are isomorphic is reduced to the case of irreducible ones. We also describes the automorphism group of a general Coxeter group in terms of those of its irreducible components.

key words: Coxeter groups; indecomposability; Isomorphism Problem; automorphism groups; centralizers. AMS Subject Classification (2000): 20F55; 20E34; 20F28.

1 Introduction

In this paper, we prove that all infinite irreducible Coxeter groups are directly indecomposable as abstract groups (Theorem 3.3).

Regarding direct indecomposability of Coxeter groups, it is well known that there exist finite irreducible Coxeter groups which are directly decomposable (such as the Weyl group G_{2}). On the other hand, for infinite irreducible Coxeter groups, no general result has been known until recently. In a recent paper [9], L. Paris proved the direct indecomposability of all infinite irreducible Coxeter groups of finite rank, by using certain special elements called essential elements which are used also in [6]. However, by definition, a Coxeter group of infinite rank never possesses an essential element, so that the proof cannot be applied directly to the case of infinite ranks.

Our result here is obtained by a different approach. Let W be an irreducible Coxeter group whose order is infinite, possibly of infinite rank. We give a complete description of the centralizer C of any normal subgroup N of W which is generated by involutions (Theorem 3.1). From the description it follows that, unless N = {1} or C = {1}, there is a subgroup H ⊊ W which contains both N and C. Once this is proved, the direct indecomposability of W is clear, since any direct factor of W is a normal subgroup and is generated by involutions (since it is a quotient of W), and its centralizer contains the complementary factor.

As a consequence of the direct indecomposability of infinite irreducible Coxeter groups, we give results on the isomorphisms between two Coxeter groups (Theorem 3.4). Since we also know how each finite irreducible Coxeter group decomposes into directly indecomposable factors, our results imply that we can determine whether or not two given Coxeter groups are isomorphic if we can determine which infinite irreducible Coxeter groups are isomorphic. In addition, our results also give certain decompositions of an automorphism of a general Coxeter group W (Theorem 3.10). One decomposition describes its form from the viewpoint of the directly indecomposable decomposition of W; another decomposition describes its form from the viewpoint of the decomposition W = W_{fin} × W_{inf}, where W_{fin} (resp. W_{inf}) is the product of the finite (resp. infinite) irreducible components of W in the given Coxeter system. Note that these results can also be deduced from the Krull-Remak-Schmidt Theorem in group theory, if the Coxeter group has a composition series. Theorem 3.4 is also a generalization of Theorem 2.1 of [9]; our proof here is similar to, but slightly more delicate than that in [9], by the lack of finiteness of the ranks. Note also that, in another recent paper [7], M. Mihalik, J. Ratcliffe and S. Tschantz also examined the “Isomorphism Problem” (namely, the problem of deciding which Coxeter groups are isomorphic) for the case of finite ranks, by a highly different approach.

Contents. Section 2 collects the preliminary facts and results. In Section 2.1, we give some remarks on general groups, especially on the definition and properties of the core subgroups. Sections 2.2 and 2.3 summarize definitions, notations and properties of Coxeter systems, Coxeter graphs and root systems of Coxeter groups. In Section 2.4, we recall a method, given by V. V. Deodhar [2], for decomposing the longest element of any finite parabolic subgroup into pairwise commuting reflections. Owing to this decomposition, we can compute easily the action of the longest element on a root, even if it is not contained in the root system of the parabolic subgroup. As an application, in Section 2.5, we determine all irreducible Coxeter groups of which the center is a nontrivial direct factor. (This is not a new result, but is included there since the result is used in the following sections.) Some properties of normalizers of parabolic subgroups are summarized as Section 2.6.

Our main results are stated and proved in Section 3. The direct indecomposability of infinite irreducible Coxeter groups is shown in Section 3.1 (Theorem 3.3). Note that the theorem also includes the description, which has been known, of all nontrivial direct product decompositions of finite irreducible Coxeter groups. In Section 3.2, we reduce the Isomorphism Problem of general Coxeter groups to the case of infinite irreducible ones (Theorem 3.4). In the proof, we give a result on such a problem in a slightly wider context (Theorem 3.9), by which our result is deduced. Moreover, another result in Section 3.3 describes the automorphism group of a general Coxeter group in terms of those of the irreducible components (Theorem 3.10 (ii)). Note that a Coxeter group possesses some ‘natural’ automorphisms, which arise from automorphisms of the irreducible components together with a permutation of isomorphic components. We also give a characterization of Coxeter groups for which the group of the ‘natural’ automorphisms has finite index in the whole automorphism group (Theorem 3.10 (iii)).

Our proof of Theorem 3.3 is based on our description of the centralizers of the normal subgroups, which are generated by involutions, in irreducible Coxeter groups (Theorem 3.1). This theorem is proved in Section 4.1 by using a description, given in Sections 4.2–4.4, of the core subgroups of normalizers of parabolic subgroups.

Acknowledgement. I would like to express my deep gratitude to everyone who helped me, especially to Professors Itaru Terada and Kazuhiko Koike for their precious advice and encouragement, and also to the referee for the careful examination and several important suggestions for improvement.

2 Preliminaries

2.1 Notes on general groups

In this paper, we treat two kinds of direct products of groups G_{λ} with (possibly infinite) index set Λ; the complete direct product (whose elements (g_{λ})_{λ} are all the maps Λ → ⨆_{μ∈Λ} G_{μ}, λ ↦ g_{λ} such that g_{λ} ∈ G_{λ}) and the restricted direct product (consisting of all the elements (g_{λ})_{λ} such that g_{λ} is the unit element of G_{λ} for all but finitely many λ ∈ Λ). Note that these two products coincide if |Λ| < ∞. Since here we treat mainly the latter type rather than the former one, we let the term “direct product” alone and the symbol ∏ mean the restricted direct product throughout this paper. (The complete one also appears in this paper, always together with notification.)

For two groups G, G′, let Hom(G, G′), Isom(G, G′) denote the sets of all homomorphisms, isomorphisms G → G′ respectively. Put End(G) = Hom(G, G) and Aut(G) = Isom(G, G). The following lemma is easy, but will be referred later.

Lemma 2.1. Assume that the center Z(G) of a group G is either trivial or a cyclic group of prime order. Then the following three conditions are equivalent: (I) Z(G) = 1 or Z(G) is not a direct factor of G. (II) If f ∈ Hom(G, Z(G)), then f(Z(G)) = 1. (III) If G′ is a direct product of (arbitrarily many) cyclic groups of prime order and f ∈ Hom(G, G′), then f(Z(G)) = 1.

Proof. This is trivial if Z(G) = 1, so that we assume that Z(G) is a cyclic group of prime order. Note that the implication (III) ⇒ (II) is obvious. (I) ⇔ (II): If (I) is not satisfied, and G = Z(G) × H, then the projection G → Z(G) does not satisfy the conclusion of (II). Conversely, if f ∈ Hom(G, Z(G)) and f(Z(G)) ≠ 1, then f(Z(G)) = Z(G), ker f ∩ Z(G) = 1 (since Z(G) is simple) and so we have G = Z(G) × ker f. (II) ⇒ (III): This is clear if G′ itself is a cyclic group of prime order (by noting that Hom(ℤ/pℤ, ℤ/ℓℤ) = 1 for distinct primes p, ℓ). For a general case, apply it to the composite map π ∘ f for every projection π from G′ to one of its factors.

Here we define the following multiplication for the set Hom(G, Z(G)) by which it forms a monoid. First, we define a map Hom(G, Z(G)) → End(G), f ↦ f^{♭} by

f^{♭}(w) = wf(w)^{−1} for all w ∈ G.

This is well defined since Z(G) is central in G. The image of H ⊂ Hom(G, Z(G)) by the map is denoted by H^{♭}. Now define the product f ∗ g of two elements f, g ∈ Hom(G, Z(G)) by

(f ∗ g)(w) = f(w)g(w)(f ∘ g)(w)^{−1} for all w ∈ G.

This is also well defined, and then Hom(G, Z(G)) forms a monoid with the trivial map (denoted by 1) as the unit element (for example, we have the associativity

\begin{aligned}((f & ∗ g) ∗ h)(w) = (f ∗ (g ∗ h))(w) \\ & = f(w)g(w)h(w)(f ∘ g)(w)^{−1}(f ∘ h)(w)^{−1}(g ∘ h)(w)^{−1}(f ∘ g ∘ h)(w) \\ & = (f ∗ h)(w)(f^{♭} ∘ g ∘ h^{♭})(w) \end{aligned} (2.1)

for f, g, h ∈ Hom(G, Z(G))). Let Hom(G, Z(G))^{×} denote the group of invertible elements of Hom(G, Z(G)) with respect to the multiplication ∗. On the other hand, End(G) also forms a monoid with composition of maps as multiplication; then the group of invertible elements in the monoid End(G) is precisely the group Aut(G).

Moreover, Aut(G) acts on the monoids Hom(G, Z(G)) and End(G) by

h · f = h ∘ f ∘ h^{−1} for h ∈ Aut(G), f ∈ Hom(G, Z(G)) or End(G).

Lemma 2.2. (i) The map f ↦ f^{♭} is an injective homomorphism Hom(G, Z(G)) → End(G) of monoids compatible with the action of Aut(G). (ii) For f ∈ Hom(G, Z(G)), the following three conditions are equivalent:

(I) f ∈ Hom(G, Z(G))^{×}. (II) f^{♭} ∈ Aut(G).

(III) The restriction f^{♭}|_{Z(G)} is an automorphism of Z(G). (iii) If H ⊂ Hom(G, Z(G))^{×} is a subgroup invariant under the action of Aut(G), then its image H^{♭} is a normal subgroup of Aut(G).

Proof. The claim (i) is straightforward, while (iii) follows from (i), (ii) and definition of the action of Aut(G). From now, we prove (ii). The implication (I) ⇒ (II) is obvious. On the other hand, (II) implies (III) since any automorphism preserves the center. Moreover, if (III) is satisfied, then we can construct the inverse element f′ of f ∈ Hom(G, Z(G)) by f′(w) = (f^{♭}|_{Z(G)})^{−1}(f(w))^{−1} (w ∈ G); we have

(f′ ∗ f)(w) = f′(w)f(w)f′(f(w))^{−1} = f′(wf(w)^{−1})f(w)
= (f^{♭}|_{Z(G)})^{−1}(f(wf(w)^{−1}))^{−1}f(w)
= (f^{♭}|_{Z(G)})^{−1}(f^{♭}(f(w)))^{−1}f(w)
= f(w)^{−1}f(w) = 1,

so that f′ ∗ f = 1. Similarly, we have f ∗ f′ = 1. Hence the claim holds.

Lemma 2.3. If a group G is abelian, then the embedding Hom(G, Z(G)) → End(G), f ↦ f^{♭}, is an isomorphism with inverse map f ↦ f^{♭}. Moreover, its restriction is an isomorphism Hom(G, Z(G))^{×} → Aut(G).

Proof. Note that Z(G) = G, so that Hom(G, Z(G)) = End(G) as sets. Thus the map End(G) → Hom(G, Z(G)), f ↦ f^{♭} is well defined. Now we have (f^{♭})^{♭}(w) = wf^{♭}(w)^{−1} = f(w) for all f ∈ End(G) and w ∈ G, so that (f^{♭})^{♭} = f. Thus the first claim holds. Now the second one follows from Lemma 2.2 (ii).

Note that, if G = G_{1} × G_{2}, then the sets Hom(G_{i}, Z(G)) (i = 1, 2) are embedded into Hom(G, Z(G)) via the map f ↦ f ∘ π_{i} (where π_{i} is the projection G → G_{i}). It is easily checked that, if f, g ∈ Hom(G_{i}, Z(G)), then f ∗ g ∈ Hom(G_{i}, Z(G)), thus Hom(G_{i}, Z(G)) forms a submonoid of Hom(G, Z(G)). Moreover, the above formula of the inverse element f′ of f ∈ Hom(G, Z(G)) implies that, if f ∈ Hom(G_{i}, Z(G)) is invertible as an element of Hom(G, Z(G)), then its inverse belongs to Hom(G_{i}, Z(G)). Thus the notation Hom(G_{i}, Z(G))^{×} is unambiguous.

Lemma 2.4. (i) Let f, g ∈ Hom(G, Z(G)) such that f(Z(G)) = g(Z(G)) = 1. Then f, g ∈ Hom(G, Z(G))^{×} and (f ∗ g)(w) = f(w)g(w) for all w ∈ G (so that f ∗ g = g ∗ f by symmetry). Moreover, the map w ↦ f(w)^{−1} is the inverse element of f in Hom(G, Z(G))^{×}. (ii) Suppose that G = G_{1} × G_{2} and Z(G_{2}) = 1. Then Hom(G, Z(G))^{×} = H_{1} ⋊ H_{2} where H_{1} = Hom(G_{2}, Z(G)), H_{2} = Hom(G_{1}, Z(G_{1}))^{×}. Moreover, H_{1} is abelian, (f ∗ g)(w) = f(w)g(w) for f, g ∈ H_{1} and f ∗ g ∗ f′ = f^{♭} ∘ g ∘ (f^{♭})^{−1} for f ∈ H_{2} and g ∈ H_{1}, where f′ is the inverse element of f ∈ H_{2}.

Proof. (i) By the hypothesis, f^{♭} is identity on Z(G), so that f is invertible by Lemma 2.2 (ii) (and g is so). The other claims follow from definition (note that now f ∘ g = 1). (ii) Note that Z(G) = Z(G_{1}) by the hypothesis. Then by (i), H_{1} is an abelian subgroup of Hom(G, Z(G))^{×} in which the multiplication is as in the statement.

For f ∈ H_{2} and g ∈ H_{1}, the formula (2.1) implies that f ∗ g ∗ f′ is as in the statement (note that f ∗ f′ = 1 and f′^{♭} = (f^{♭})^{−1}). In particular, we have f ∗ g ∗ f′(G_{1}) ⊂ f^{♭} ∘ g(G_{1}) = 1, since f′ ∈ H_{2} and so f′^{♭}(G_{1}) ⊂ G_{1}. This means that f ∗ g ∗ f′ ∈ H_{1}. Since obviously H_{1} ∩ H_{2} = 1, we have H_{1}H_{2} = H_{1} ⋊ H_{2}.

Finally, let f ∈ Hom(G, Z(G))^{×}. Take g ∈ H_{1} such that g(w) = f ∘ π_{2}(w)^{−1} where π_{2} is the projection G → G_{2} (this is the inverse element of f ∘ π_{2} ∈ H_{1}). Then for w ∈ G_{2}, we have

(g ∗ f)(w) = g(w)f(w)g(f(w))^{−1} = f(w)^{−1}f(w) = 1

since g(Z(G)) = 1. This means that g ∗ f ∈ Hom(G_{1}, Z(G_{1})), while it is invertible since both f and g are so. Thus we have g ∗ f ∈ H_{2} and f = (f ∘ π_{2}) ∗ g ∗ f ∈ H_{1}H_{2}. Hence Hom(G, Z(G))^{×} = H_{1} ⋊ H_{2}.

In the proof of our results, we use the following notion. For a group G, we write H ≤ G, H ◁ G if H is a subgroup, normal subgroup of G, respectively.

Definition 2.5. For H ≤ G, define the core Core_{G}(H) of H in G to be the unique maximal normal subgroup of G contained in H (namely, ⋂_{w∈G} wHw^{−1}).

The following properties are deduced immediately from definition:

If H_{1} ≤ H_{2} ≤ G, then Core_{G}(H_{1}) ⊂ Core_{G}(H_{2}).  (2.2)
If Core_{G}(H) ≤ H_{1} ≤ G, then Core_{G}(H) ⊂ Core_{G}(H_{1}).  (2.3)
If H_{λ} ≤ G (λ ∈ Λ), then Core_{G}( ⋂_{λ∈Λ} H_{λ}) = ⋂_{λ∈Λ} Core_{G}(H_{λ}).  (2.4)

If H_{1} ≤ H_{2} ≤ G, w ∈ G and wH_{1}w^{−1} ∩ H_{2} = 1, then H_{1} ∩ Core_{G}(H_{2}) = 1. (2.5)

Lemma 2.6. Let G_{1} ≤ G_{2} ≤ ⋯, H_{1} ≤ H_{2} ≤ ⋯ be two infinite chains of subgroups of the same group such that G_{i} ∩ H_{j} = H_{i} for all i ≤ j. Put G = ⋃_{i=1}^{∞} G_{i} and H = ⋃_{i=1}^{∞} H_{i}. Then Core_{G}(H) ⊂ ⋃_{i=1}^{∞} Core_{G_{i}}(H_{i}).

Proof. It is enough to show that Core_{G}(H)∩H_{i} ⊂ Core_{G_{i}}(H_{i}) (or more strongly, Core_{G}(H) ∩ H_{i} ◁ G_{i}) for all i. Note that the hypothesis implies G_{i} ∩ H = H_{i}. Then for g ∈ G_{i} and h ∈ Core_{G}(H) ∩ H_{i}, we have ghg^{−1} ∈ Core_{G}(H) and ghg^{−1} ∈ G_{i}, so that ghg^{−1} ∈ G_{i} ∩ H = H_{i}. Thus the claim holds.

The next lemma describes the centralizers of normal subgroups in terms of the cores of certain subgroups. Before stating this, note the following easy facts:

If H ◁ G, then the centralizer Z_{G}(H) of H is also normal in G.  (2.6)
If X_{1}, X_{2} ⊂ G are subsets and X_{1} ⊂ Z_{G}(X_{2}), then X_{2} ⊂ Z_{G}(X_{1}). (2.7)

Lemma 2.7. Let H be the smallest normal subgroup of G containing a subset X ⊂ G. Then Z_{G}(H) = Core_{G}(Z_{G}(X)) = ⋂_{x∈X} Core_{G}(Z_{G}(x)).

Proof. The second equality follows from (2.4). For the first one, the inclusion ⊂ is deduced from (2.6) (since Z_{G}(H) ⊂ Z_{G}(X)). For the other inclusion, the centralizer of Core_{G}(Z_{G}(X)) in G is normal in G (by (2.6)) and contains X, so that it also contains H. Thus the claim follows from (2.7).

2.2 Coxeter groups and Coxeter graphs

Here we refer to [5] for basic definitions and properties. A pair (W, S) of a group W and its generating set S is a Coxeter system (and W itself is a Coxeter group) if W has the presentation

W = ⟨S | (st)^{m(s,t)} = 1 if s, t ∈ S and m(s, t) < ∞⟩

where m : S × S → {1, 2, … } ∪ {∞} is a symmetric map such that m(s, t) = 1 if and only if s = t. (W, S) is said to be finite (infinite) if the group W is finite (infinite, respectively). The cardinality of S is called the rank of (W, S) (or even of W). Throughout this paper, we do not assume, unless otherwise noticed, that the rank of (W, S) is finite (or even countable). Note that, owing to the well-known fact that the element st ∈ W above has precisely order m(s, t) in W, this map m can be recovered uniquely from the Coxeter system (W, S).

Two Coxeter systems (W, S) and (W′, S′) are said to be isomorphic if there is some f ∈ Isom(W, W′) such that f(S) = S′. Then there is a one-to-one correspondence (up to isomorphism) between Coxeter systems and the Coxeter graphs; which are simple (loopless), undirected, edge-labelled graphs with labels in {3, 4, … }∪{∞}. The Coxeter graph Γ corresponding to (W, S) has the vertex set S, and two vertices s, t ∈ S are joined in Γ by an edge with label m(s, t) if and only if m(s, t) ≥ 3 (by convention, the labels ‘3’ are usually omitted). Γ (or (W, S)) is said to be of finite type if W is finite. It is also well known that, when W_{I} denotes the parabolic subgroup of W generated by a subset I ⊂ S, (W_{I}, I) is also a Coxeter system with Coxeter graph Γ_{I} which is the full subgraph of Γ on the vertex set I.

A Coxeter system (W, S) is called irreducible if the corresponding Coxeter graph Γ is connected. In this case, W is also said to be irreducible. As is well known, W is decomposed as the direct product of its irreducible components, which are the parabolic subgroups W_{I} of W corresponding to the connected components Γ_{I} of Γ (in this case, each subset I is also said to be an irreducible component of S). A parabolic subgroup W_{I} ⊂ W is said to be irreducible if the Coxeter system (W_{I}, I) is irreducible. As we mentioned in Introduction, an irreducible Coxeter group may be directly decomposable (as an abstract group) in general. Our main result determines which irreducible Coxeter group is indeed directly indecomposable.

In this paper, we use the following notations for some Coxeter graphs.

Definition 2.8. We use the notations in Fig. 1. For each of the Coxeter graphs, let s_{i} denote the vertex having label i. Moreover, for each Coxeter graph Γ (T_{n}) in Fig. 1 (T = A, B, D, E, F, H), let Γ (T_{k}) (k < n) be the full subgraph of Γ (T_{n}) on vertex set {s_{i} | 1 ≤ i ≤ k}. For any T , let (W (T ), S(T )) be the Coxeter system corresponding to the Coxeter graph Γ (T ).

Γ (A_{∞}) = ∘ ∘ ∘ ∘  ⋯ 1 2 3 4  Γ (B_{∞}) = ∘ ∘ ∘ ∘  ⋯ 1 2 3 4 4

[figure: Γ (D_{∞}) =; ∘; ∘; ∘; ∘; ∘; ⋯; 1; 2; 3; 4; 5; Γ (E_{8}) =; ∘; ∘; ∘; ∘; ∘; ∘; ∘; ∘; 1; 3; 4; 5; 6; 7; 8; 2]

Γ (F_{4}) = ∘ ∘ ∘ ∘ 1 2 3 4 4  Γ (H_{4}) = ∘ ∘ ∘ ∘ 1 2 3 4 5  Γ (I_{2}(m)) = ∘ ∘ 1 2 m

Γ (A_{∞,∞}) =  ∘ ∘ ∘ ∘ ∘ ∘ ∘ ⋯  ⋯ −3 −2 −1 0 1 2 3  ⊃ Γ (A_{∞})

Figure 1: Some connected Coxeter graphs

By definition, Γ (T_{∞}) (T = A, B, D) and Γ (A_{∞,∞}) are Coxeter graphs with countable (infinite) vertex sets. On the other hand, it is well known that the Coxeter graphs Γ (A_{n}) (1 ≤ n < ∞), Γ (B_{n}) (2 ≤ n < ∞), Γ (D_{n}) (4 ≤ n < ∞), Γ (E_{6}), Γ (E_{7}), Γ (E_{8}), Γ (F_{4}), Γ (H_{3}), Γ (H_{4}) and Γ (I_{2}(m)) (5 ≤ m < ∞) are all the connected Coxeter graphs of finite type (up to isomorphism). Note that Γ (B_{1}) = Γ (D_{1}) = Γ (A_{1}), while Γ (D_{2}) ≃ Γ (A_{1} × A_{1}) and Γ (D_{3}) ≃ Γ (A_{3}) (but the vertex labels are different).

2.3 Root systems of Coxeter groups

For a Coxeter system (W, S), let Π be the set of symbols α_{s} (s ∈ S) and V the vector space over ℝ containing the set Π as a basis. We define the symmetric bilinear form ⟨ , ⟩ on V for the basis by

⟨α_{s}, α_{t}⟩ = − cos(π/m(s, t)) if m(s, t) < ∞, ⟨α_{s}, α_{t}⟩ = −1 if m(s, t) = ∞.

Then W acts faithfully on the space V by s · v = v − 2⟨α_{s}, v⟩α_{s} (s ∈ S, v ∈ V). Let Φ = W · Π, the root system of (W, S). The above rule implies that the action of W preserves the bilinear form; as a consequence, any element (root) of Φ is a unit vector. It is a crucial fact that Φ is a disjoint union of the set Φ^{+} of positive roots (i.e. roots in which the coefficient of every α_{s} ∈ Π is ≥ 0) and the set Φ^{−} = −Φ^{+} of negative roots. It is well known that the cardinality of the set Φ [w] = {γ ∈ Φ^{+} | w · γ ∈ Φ^{−}} is (finite and) equal to the length ℓ(w) of w ∈ W with respect to the generating set S. From the fact it follows easily that the set Φ [w] characterizes the element w ∈ W; namely,

if w, u ∈ W and Φ [w] = Φ [u] , then w = u  (2.8)

(observe Φ [wu^{−1}] = ∅ and so wu^{−1} = 1).

The reflection along a root γ = w · α_{s} ∈ Φ is defined by s_{γ} = wsw^{−1} ∈ W. This definition does not depend on the choice of w and s, and s_{γ} indeed acts as a reflection on the space V; s_{γ} · v = v − 2⟨γ, v⟩γ for v ∈ V. Note that s_{α_{s}} = s for s ∈ S. The following fact is easy to show (by the fact that Φ = Φ^{+} ⊔ Φ^{−}):

if s ∈ S, γ ∈ Φ^{+} and ⟨α_{s}, γ⟩ > 0, then s_{γ} · α_{s} ∈ Φ^{−}.  (2.9)

For v ∈ V, put

v = ∑_{s∈S}([α_{s}] v)α_{s} and supp(v) = {s ∈ S | [α_{s}] v ≠ 0}.

For I ⊂ S, let V_{I} be the subspace of V spanned by the set Π_{I} = {α_{s} | s ∈ I} and Φ_{I} = Φ ∩ V_{I} (namely, the set of all γ ∈ Φ such that supp(γ) ⊂ I). Then it is well known that Φ_{I} coincides with the root system W_{I} · Π_{I} of the Coxeter system (W_{I}, I) (cf. Lemma 4 of [3], etc. for the proof). This fact yields the following:

If γ ∈ Φ, then (γ ∈ Φ_{supp(γ)} and so) the set supp(γ) is connected in Γ.

(2.10)

Moreover, it is well known (cf. [5], Section 5.8, Exercise 4, etc.) that:

If I ⊂ S and γ ∈ Φ, then s_{γ} ∈ W_{I} if and only if γ ∈ Φ_{I}.  (2.11)

For I ⊂ S, let

I^{⊥} = {s ∈ S ∖ I | st = ts for all t ∈ I}

= {s ∈ S ∖ I | s is adjacent in Γ to no element of I} = {s ∈ S | α_{s} is orthogonal to every α_{t} ∈ Π_{I}}.

Then we have the following properties:

If γ ∈ Φ^{+} and supp(γ) ⊄ I ⊂ S, then w · γ ∈ Φ^{+} for all w ∈ W_{I}. (2.12)

If γ ∈ Φ, I = supp(γ) and s ∈ S ∖ (I ∪ I^{⊥}), then supp(s · γ) = I ∪ {s}. (2.13)

(For (2.12), take some t ∈ supp(γ) ∖ I, then w · γ has the same (positive) coefficient of α_{t} as γ. For (2.13), note that ⟨α_{s}, γ⟩ < 0 by the hypothesis.)

For I ⊂ S and w ∈ W, let Φ_{I}^{+} = Φ_{I} ∩ Φ^{+}, Φ_{I}^{−} = Φ_{I} ∩ Φ^{−} and Φ_{I} [w] = Φ_{I} ∩ Φ [w].

Lemma 2.9. Let w ∈ W, I, J ⊂ S and suppose that I ∩ J = ∅, w · Π_{I} = Π_{I} and w · Π_{J} ⊂ Φ^{−}. Then Φ_{I∪J} [w] = Φ_{I∪J}^{+} ∖ Φ_{I}.

Proof. Let γ ∈ Φ_{I∪J}^{+} such that [α_{s}] γ > 0 for at least one s ∈ J (note that w · α_{s} ∈ Φ^{−}). Now if w · α_{s} ∈ Φ_{I}^{−}, then α_{s} = w^{−1} · (w · α_{s}) must be a linear combination of Π_{I} (since w · Π_{I} = Π_{I}), but this is impossible. Thus we have [α_{t}] (w · α_{s}) < 0 for some t ∈ S ∖ I. Moreover, the hypothesis implies that [α_{t}] (w · α_{s′}) = 0 for all s′ ∈ I and [α_{t}] (w · α_{s′}) ≤ 0 for all s′ ∈ J. Thus we have

\begin{aligned}[α_{t}] (w · γ) & = [α_{t}] (w · ∑_{s′∈I∪J}([α_{s′}] γ)α_{s′}) \\ & = ∑_{s′∈I∪J}([α_{s′}] γ) [α_{t}] (w · α_{s′}) ≤ [α_{s}] γ [α_{t}] (w · α_{s}) < 0. \end{aligned}

Hence the claim holds, since w · Φ_{I}^{+} ⊂ Φ^{+} by the hypothesis.

Definition 2.10. For a Coxeter system (W, S), we define the odd Coxeter graph Γ^{odd} of (W, S) to be the subgraph of Γ obtained by removing all edges labelled by an even number or ∞.

It is well known (cf. [5], Section 5.3, Exercise, etc.) that, for s, t ∈ S,

α_{t} ∈ W · α_{s} if and only if s, t are in the same connected component of Γ^{odd}.

(2.14)

Moreover, the following lemma is deduced immediately from the definition that all fundamental relations of W are of the form (st)^{m(s,t)} = 1 (s, t ∈ S).

Lemma 2.11. Any f ∈ Hom(W, {±1}) assigns the same value to every vertex s ∈ S of a connected component of Γ^{odd}. Conversely, any mapping S → {±1} having this property extends uniquely to a homomorphism W → {±1}.

2.4 Reflection decompositions of longest elements

If W_{I} is a finite parabolic subgroup of a Coxeter group W, then let w_{0}(I) denote the longest element of W_{I}. This element is an involution and maps the set Π_{I} onto −Π_{I}, so that there is an involutive graph automorphism σ_{I} of the Coxeter graph Γ_{I} such that

w_{0}(I) · α_{s} = −α_{σ_{I}(s)} for all s ∈ I.

It is well known that, for an irreducible Coxeter system (W, S), we have Z(W) ≠ 1 if and only if W ≃ W (T ) for one of T = A_{1}, B_{n} (n < ∞), D_{k} (k ≥ 4 even), E_{7}, E_{8}, F_{4}, H_{3}, H_{4} and I_{2}(m) (m ≥ 6 even). This condition is also equivalent to that |W | < ∞ and σ_{S} = id_{S}. Moreover, Z(W) = {1, w_{0}(S)} if Z(W) ≠ 1, while σ_{S} is determined as the unique non-identical automorphism of Γ whenever W is finite, irreducible and Z(W) = 1. Note that any automorphism τ ∈ Aut(Γ) induces naturally an automorphism of W, which maps each element w_{0}(I) to w_{0}(τ(I)).

The next lemma introduces certain normal subgroups G_{B_{n}}, G_{D_{n}} of W (B_{n}), W (D_{n}), respectively, which play an important role in later sections.

Lemma 2.12. (See Definition 2.8 for notations.)

(i) Let 1 ≤ n ≤ ∞. Then the subgroup G_{B_{n}} of W (B_{n}) generated by all w_{0}(S(B_{i})) (1 ≤ i ≤ n, i < ∞) is normal in W (B_{n}).

(ii) Let 1 ≤ n ≤ ∞. Then the smallest normal subgroup G_{D_{n}} of W (D_{n}) containing all w_{0}(S(D_{2k})) (1 ≤ k < ∞, 2k ≤ n) is the subgroup generated by all w_{0}(S(D_{i})) (2 ≤ i ≤ n, i < ∞). (iii) Moreover, each of the above normal subgroups is an elementary abelian 2-group with the generating set given there as the basis.

Proof. For the case n < ∞, we refer to Section 2.10 of [5] the following realizations of W (B_{n}) and W (D_{n}). Namely, W (B_{n}) is isomorphic to the semidirect product K_{n} ⋊ Sym_{n} of an elementary abelian 2-group K_{n} ≃ {±1}^{n}, where the i-th element of the basis is denoted by κ_{i}, by the group Sym_{n} of permutations of the κ_{i}’s. The isomorphism sends s_{1} to κ_{1} and s_{i} (i ≥ 2) to (i−1 i) ∈ Sym_{n}. Now it is easily checked that w_{0}(S(B_{i})) = κ_{1}κ_{2} ⋯ κ_{i}, so that we have G_{B_{n}} = K_{n}. Moreover, W (D_{n}) is isomorphic to K_{n}^{+} ⋊ Sym_{n} where K_{n}^{+} is the subgroup of K_{n} generated by κ_{i}κ_{i+1} (1 ≤ i ≤ n − 1), and the isomorphism sends s_{1} to κ_{1}κ_{2} · (1 2) and s_{i} (i ≥ 2) to (i − 1 i). Now w_{0}(S(D_{i})) = κ_{1}κ_{2} ⋯ κ_{i} if i ≥ 2 is even and w_{0}(S(D_{i})) = κ_{2}κ_{3} ⋯ κ_{i} if i ≥ 2 is odd, so that we have G_{D_{n}} = K_{n}^{+}. By those observations, the claims (i)–(iii) are deduced immediately.

For the remaining case n = ∞, note that W (B_{∞}) is the direct limit of the sequence W (B_{1}) ⊂ W (B_{2}) ⊂ ⋯, and similarly for W (D_{∞}). Thus the claims are also deduced in this case by the argument in the previous paragraph.

In the paper [2], Deodhar established a method (in the proof of Theorem 5.4) for decomposing any involution w ∈ W as a product of commuting reflections. From now, we apply this method and then obtain a decomposition of any longest element w_{0}(I), which we call here a reflection decomposition. Among the various expressions of the elements w_{0}(I), this decomposition possesses an advantage in a computation of the action on the space V. First, to each finite irreducible Coxeter system (W, S) = (W (T ), S(T )) of type T , we associate a (or two) positive root(s) α̃_{T} = α_{T}^{(1)} (and α_{T}^{(2)}), as follows (where we abbreviate c_{1}α_{1} + c_{2}α_{2} + ⋯ + c_{n}α_{n} ∈ V to (c_{1}, c_{2}, …, c_{n}) in some cases):

α̃_{A_{n}} = ∑_{i=1}^{n} α_{i} (1 ≤ n < ∞), α̃_{D_{n}} = α_{1} + α_{2} + ∑_{i=3}^{n−1} 2α_{i} + α_{n} (4 ≤ n < ∞),

α_{B_{n}}^{(1)} = α_{1} + ∑_{i=2}^{n} \sqrt{2} α_{i}, α_{B_{n}}^{(2)} = \sqrt{2} α_{1} + ∑_{i=2}^{n−1} 2α_{i} + α_{n} (2 ≤ n < ∞),

α̃_{E_{6}} = (1, 2, 2, 3, 2, 1), α̃_{E_{7}} = (2, 2, 3, 4, 3, 2, 1), α̃_{E_{8}} = (2, 3, 4, 6, 5, 4, 3, 2),

α_{F_{4}}^{(1)} = (2, 3, 2\sqrt{2} , \sqrt{2} ), α_{F_{4}}^{(2)} = (\sqrt{2} , 2\sqrt{2} , 3, 2),

α̃_{H_{3}} = (c + 1, 2c, c), α̃_{H_{4}} = (3c + 2, 4c + 2, 3c + 1, 2c) (where c = 2 cos \frac{π}{5} ),

α̃_{I_{2}(m)} = \frac{1}{2 sin(π/2m)}α_{1} + \frac{1}{2 sin(π/2m)}α_{2} (m ≥ 5 odd),

α_{I_{2}(m)}^{(i)} = \frac{cos(π/m)}{sin(π/m)}α_{i} + \frac{1}{sin(π/m)}α_{3−i} (m ≥ 5 even, i = 1, 2).

To check that each of these is actually a root of (W (T ), S(T )), note the equality c^{2} = c + 1 and the following formula for the root system of type I_{2}(m):

If w = (⋯ s_{2}s_{1}s_{2}) ∈ W (I_{2}(m)) (k elements), then

w · α_{1} = \begin{cases}\frac{sin(kπ/m)}{sin(π/m)}α_{1} + \frac{sin((k+1)π/m)}{sin(π/m)}α_{2} & if k is odd, \\ \frac{sin((k+1)π/m)}{sin(π/m)}α_{1} + \frac{sin(kπ/m)}{sin(π/m)}α_{2} & if k is even. \end{cases}

For example, we have

α_{F_{4}}^{(1)} = s_{1}s_{2}s_{3}s_{4}s_{2}s_{3}s_{2} · α_{1}, α_{F_{4}}^{(2)} = s_{4}s_{3}s_{2}s_{1}s_{3}s_{2}s_{3} · α_{4},

α̃_{H_{3}} = s_{2}s_{1}s_{2}s_{1}s_{3}s_{2} · α_{1}, α̃_{H_{4}} = s_{4}s_{3}s_{2}s_{1}s_{2}s_{1}s_{3}s_{2}s_{1}s_{4}s_{3}s_{2}s_{1}s_{2}s_{3}s_{4} · α̃_{H_{3}},

α̃_{I_{2}(2k+1)} = (⋯ s_{2}s_{1}s_{2}) · α_{1} (k elements), α_{I_{2}(4k)}^{(i)} = (s_{3−i}s_{i})^{k−1}s_{3−i} · α_{i}.

By (2.14), if T ≠ B_{n}, F_{4}, I_{2}(m) (m even), then Φ consists of a single orbit W (T ) · α_{1} (and so it contains α̃_{T}). On the other hand, if T = B_{n}, F_{4} or I_{2}(4k), then (2.14) implies that Φ consists of two orbits (namely, W · α_{1} and W · α_{2} if T = B_{n}, I_{2}(4k), and W · α_{1} and W · α_{4} if T = F_{4}). In these case, α_{T}^{(1)} lies in the orbit W · α_{1} and α_{T}^{(2)} lies in the other one.

In contrast with the above cases, if T = I_{2}(4k + 2), then Φ consists of two orbits W (T ) · α_{1} and W (T ) · α_{2}, and now we have α_{T}^{(1)} ∈ W (T ) · α_{2} (and α_{T}^{(2)} lies in the other orbit). In fact, we have α_{I_{2}(4k+2)}^{(i)} = (s_{3−i}s_{i})^{k} · α_{3−i} for i = 1, 2.

To simplify the description, we denote the reflection along the root α_{T}^{(i)} by r̃(T , i). If we have only one root α_{T}^{(i)}, namely T ≠ B_{n}, F_{4}, I_{2}(m) (m even), then we also write r̃(T ) = r̃(T , 1).

Remark 2.13. By the above observation, if T = B_{n}, F_{4} or I_{2}(4k), then r̃(T , 1) is conjugate to s_{1}, and r̃(T , 2) is conjugate to s_{2} (if T = B_{n} or I_{2}(4k)) or to s_{4} (if T = F_{4}). On the other hand, if T = I_{2}(4k + 2), then r̃(T , 1), r̃(T , 2) are conjugate to s_{2}, s_{1}, respectively.

Lemma 2.14. (i) If T ≠ A_{n} (n ≥ 2), I_{2}(m) (m odd), then for the root α_{T}^{(i)}, there is an index N(T , i) such that ⟨α_{T}^{(i)}, α_{j}⟩ = 0 for all j ≠ N(T , i). Moreover, we have ⟨α_{T}^{(i)}, α_{N(T ,i)}⟩ > 0 and Φ [r̃(T , i)] = Φ^{+} ∖ Φ_{S(T )∖{s_{N(T ,i)}}}. (If we have

only one root α_{T}^{(i)}, then we also write N(T ) = N(T , 1).) (ii) If T = A_{n} (n ≥ 2) or I_{2}(m) (m odd), then there are two indices N_{1}(T ), N_{2}(T ) such that ⟨α̃_{T} , α_{N_{j}(T )}⟩ > 0 for j = 1, 2 and ⟨α̃_{T} , α_{j}⟩ = 0 for all j ≠ N_{1}(T ), N_{2}(T ). Moreover, we have Φ [r̃(T )] = Φ^{+} ∖ Φ_{S(T )∖{s_{N_{1}(T )},s_{N_{2}(T )}}}.

Proof. (i) The first claim follows from a direct computation, by putting

N(A_{1}) = 1, N(B_{n}, 1) = n, N(B_{n}, 2) = n − 1, N(D_{n}) = n − 1, N(E_{6}) = 2, N(E_{7}) = 1, N(E_{8}) = 8, N(F_{4}, 1) = 1, N(F_{4}, 2) = 4, N(H_{3}) = 2, N(H_{4}) = 4, N(I_{2}(2k), 1) = 2, N(I_{2}(2k), 2) = 1.

For the second one, expand the equality ⟨α_{T}^{(i)}, α_{T}^{(i)}⟩ = 1 and use the first claim. Now the third one follows from (2.9) and Lemma 2.9. (ii) The former claim also follows from a direct computation, by putting

N_{1}(A_{n}) = 1, N_{2}(A_{n}) = n, N_{1}(I_{2}(2k + 1)) = 1, N_{2}(I_{2}(2k + 1)) = 2.

The remaining proof is similar to (i).

Now our method for obtaining a reflection decomposition of w_{0}(I), which is the same as a decomposition given by Deodhar’s method ([2], proof of Theorem 5.4), is summarized as follows:

(I) If I = ∅, then this algorithm finishes with the (trivial) decomposition w_{0}(I) = 1. If I ≠ ∅, choose an irreducible component J of I. Let J = S(T ).

(II) If T ≠ A_{n} (n ≥ 2), I_{2}(m) (m odd), take the (or one of the two) root(s) α_{T}^{(i)}. By Lemma 2.14 (i), r̃(T , i) commutes with all elements of K = I ∖ {s_{N(T ,i)}}, and we have w_{0}(I) = r̃(T , i)w_{0}(K) (since Φ [w_{0}(I)] = Φ [r̃(T , i)w_{0}(K)] = Φ_{I}^{+}; cf. (2.8)). Then apply this algorithm inductively to the (smaller) set K.

(III) If T = A_{n} (n ≥ 2) or I_{2}(m) (m odd), then similarly, r̃(T ) commutes with all elements of K = I ∖ {s_{N_{1}(T )}, s_{N_{2}(T )}} and w_{0}(I) = r̃(T )w_{0}(K) by Lemma 2.14 (ii). Then apply this algorithm inductively to the (smaller) set K.

By collecting the subset K ⊂ I appearing in the step (II) or (III) of every turn, we obtain a decreasing sequence (K_{0} = I,) K_{1}, …, K_{r−1}, K_{r} = ∅. We call this a generator sequence (of length r) for the set I.

2.5 Direct product decompositions of finite Coxeter groups

As an application of the reflection decomposition introduced in Section 2.4, we determine easily which finite irreducible Coxeter groups have the center as a nontrivial direct factor. Although this result itself is not a new one, we restate it here since the result is used in later sections.

For a Coxeter system (W, S), let W^{+} denote the normal subgroup of W (of index two) consisting of elements of even length. This coincides with the kernel of the map sgn ∈ Hom(W, {±1}) such that sgn(w) = (−1)^{ℓ(w)}. Since any reflection in W has odd length, the following lemma follows from (the proof of) Lemma 2.1:

Lemma 2.15. If (W, S) is a finite irreducible Coxeter system and Z(W) ≠ 1, then we have W = Z(W) × W^{+} if and only if some (or equivalently, any) generator sequence for S (cf. Section 2.4) has odd length.

Theorem 2.16. Let (W, S) be an irreducible Coxeter system such that Z(W) ≠ 1 (so that |W | < ∞). Then Z(W) (≃ W (A_{1})) is a proper direct factor of W if and only if W ≃ W (T ) for T = B_{2k+1}, I_{2}(4k + 2) (k ≥ 1), E_{7} or H_{3}. In the first two cases, W is isomorphic to W (A_{1})×W (D_{2k+1}), W (A_{1})×W (I_{2}(2k+1)) respectively. In the last two cases, we have W = Z(W) × W^{+}.

Proof. Note that Z(W) ≃ {±1} by the hypothesis. Since Z(W (A_{1})) = W (A_{1}), we may assume W ≠ W (A_{1}).

Case 1. W = W (B_{n}) (n ≥ 2): First, we have Hom(W, {±1}) = {1, sgn, ε_{1}, ε_{2}} by Lemma 2.11, where 1 denotes the trivial map, ε_{1}(s_{1}) = −1, ε_{1}(s_{i}) = 1, ε_{2}(s_{1}) = 1 and ε_{2}(s_{i}) = −1 (i ≠ 1). Now we consider the following reflection decomposition:

w_{0}(S) = r̃(B_{n}, 1)r̃(B_{n−1}, 1) ⋯ r̃(B_{2}, 1)s_{1}.

By Remark 2.13, each reflection r̃(B_{k}, 1) is conjugate to s_{1}. This implies that any expression of r̃(B_{k}, 1) as a product of generators contains an odd number of s_{1} and an even number of s_{i} (i ≠ 1). Thus we have

sgn(r̃(B_{k}, 1)) = ε_{1}(r̃(B_{k}, 1)) = −1 and ε_{2}(r̃(B_{k}, 1)) = 1.

If n is even, then all f ∈ Hom(W, {±1}) maps w_{0}(S) to 1 by the above property. Thus by Lemma 2.1, Z(W) is not a direct factor.

On the other hand, if n is odd, then we have ε_{1}(w_{0}(S)) = −1 and so W = Z(W) × ker ε_{1} by the proof of Lemma 2.1. Note that ker ε_{1} consists of elements in which s_{1} appears an even number of times. Since s_{1} commutes with all s_{i} (3 ≤ i ≤ n), it can be deduced directly that ker ε_{1} is generated by s′_{1} = s_{1}s_{2}s_{1} and all s′_{i} = s_{i} (2 ≤ i ≤ n). Moreover, ker ε_{1} forms a Coxeter group of type D_{n}; in fact, s′_{1}, …, s′_{n} satisfy the fundamental relations of type D_{n} (so that ker ε_{1} is a quotient of W (D_{n})), while the order |W (B_{n})|/2 of ker ε_{1} coincides with |W (D_{n})|. Hence the claim holds in this case.

Case 2. W = W (T ) for T = D_{2k} (k ≥ 2), E_{7}, E_{8}, H_{3}, H_{4}: Since Γ^{odd} is connected in this case, we have Hom(W, {±1}) = {1, sgn} by Lemma 2.11. Thus the claim follows from Lemmas 2.1 and 2.15, by taking the following generator sequence for S (where we abbreviate the set {s_{i_{1}}, s_{i_{2}}, …, s_{i_{r}}} to i_{1}i_{2} ⋯ i_{r}):

S(D_{2k−2}) ∪ {s_{2k}}, S(D_{2k−2}), …, S(D_{4}), 124, 12, 1, ∅ if T = D_{2k},
S(E_{7}), 234567, 23457, 2345, 235, 23, 2, ∅  if T = E_{8},
234567, 23457, 2345, 235, 23, 2, ∅  if T = E_{7},
S(H_{3}), 13, 1, ∅  if T = H_{4},
13, 1, ∅  if T = H_{3}

(note that the first sequence consists of 2k terms).

Case 3. W = W (F_{4}): We have a generator sequence 234, 23, 2, ∅ for S and the corresponding decomposition of w_{0}(S) into four reflections, all of which are conjugate to s_{1} and s_{2} (cf. Remark 2.13). This (and Lemma 2.11) implies that any f ∈ Hom(W, {±1}) maps all the four reflections to the same element f(s_{1}), so that f(w_{0}(S)) = 1. Hence the claim follows from Lemma 2.1.

Case 4. W = W (I_{2}(2k)) (k ≥ 3): We have a reflection decomposition w_{0}(S) = r̃(I_{2}(2k), 1)s_{1}. If k is even, then r̃(I_{2}(2k), 1) is conjugate to s_{1} (cf. Remark 2.13). Now by a similar argument to the previous case, any f ∈ Hom(W, {±1}) maps w_{0}(S) to 1. Thus Z(W) is not a direct factor by Lemma 2.1.

On the other hand, if k is odd, then r̃(I_{2}(2k), 1) is conjugate to s_{2} (cf. Remark 2.13). Thus ε_{1} ∈ Hom(W, {±1}) (ε(s_{1}) = −1, ε(s_{2}) = 1) sends w_{0}(S) to −1, so that W = Z(W) × ker ε_{1} by the proof of Lemma 2.1. Moreover, ker ε_{1} is generated by two reflections s_{1}s_{2}s_{1} and s_{2}, and so ker ε_{1} is a Coxeter system of type I_{2}(k) (since s_{1}s_{2}s_{1}s_{2} has order k). Hence the claim holds in all cases.

Since the groups W (E_{7})^{+} and W (H_{3})^{+} are known to be (isomorphic to) the well-examined simple groups S_{6}(2) and A_{5} respectively (cf. [5], Sections 2.12–13, etc.), we omit the proof of the following properties of these groups. Note that these properties can also be proved by using Theorems 2.16 and 3.3 below.

Lemma 2.17. Let G = W (T )^{+}, T ∈ {E_{7}, H_{3}}. Then G has trivial center, is directly indecomposable and is generated by involutions. Moreover, G is not isomorphic to a Coxeter group.

2.6 Notes on normalizers in Coxeter groups

In this subsection, we summarize some properties of normalizers N_{W} (W_{I}) of parabolic subgroups W_{I} in Coxeter groups W. Note that the explicit structure of N_{W} (W_{I}) is well-described in [1] (or in [4], when |W | < ∞) and also in [8]; however, such a strong result is not required for our purpose.

The following result is due to D. Krammer [6]. The first part appears in Proposition 3.1.9 (a) of [6], while the second one is easily deduced by the argument of Section 3.1 of [6], particularly by Corollary 3.1.5.

Proposition 2.18 (cf. [6]). (i) If I ⊂ S, then N_{W} (W_{I}) is the semidirect product W_{I} ⋊ G_{I} of W_{I} by the group G_{I} = {w ∈ W | w · Π_{I} = Π_{I}}. (ii) If I ⊂ J ⊂ S and W_{I} is an infinite irreducible component of W_{J}, then N_{W} (W_{J}) ⊂ W_{I∪I^{⊥}}.

We also require the following result. This was originally given by Deodhar [2], in the proof of Proposition 4.2, for the case |S| < ∞ only. Here we give a proof covering the case |S| = ∞ as well for the sake of completeness.

Proposition 2.19 (cf. [2], Proposition 4.2). If (W, S) is irreducible and |W | = ∞, then |Φ ∖ Φ_{I}| = ∞ for all proper subsets I ⊂ S.

Proof. We consider the case |S| < ∞ first. Put Ψ_{J} = {γ ∈ Φ | supp(γ) = J} for J ⊂ S. Our aim is to show |Ψ_{S}| = ∞. If this fails, and J ⊂ S is maximal subject to |Ψ_{J}| = ∞ (note that some of Ψ_{K}’s must be infinite since their finite union Φ is so), then J ⊊ S and s ∈ S ∖J maps Ψ_{J} injectively into Ψ_{J∪{s}} when s is adjacent to J in Γ. Since Γ is connected by the hypothesis, such an element s indeed exists, contradicting the maximality of J. Hence the claim holds.

Secondly, suppose |S| = ∞. Since Γ is connected, there are infinitely many finite subsets J ⊂ S such that J ⊄ I and Γ_{J} is connected. Now the claim holds, since each Φ_{J} contains a root γ such that supp(γ) = J.

By those properties, we can prove the following corollary.

Corollary 2.20. Let s ∈ S and I = S ∖ {s}.

(i) If 1 ≠ w ∈ G_{I}, then Φ [w] = Φ^{+} ∖ Φ_{I}. Hence by (2.8), such an element w is unique if it exists. (ii) If |W | < ∞ and w_{0}(S) ∈ N_{W} (W_{I}), then N_{W} (W_{I}) = W_{I} ⋊ {1, w_{0}(S)}. (iii) If (W, S) is irreducible and |W | = ∞, then G_{I} = 1 and N_{W} (W_{I}) = W_{I}.

Proof. (i) In this case, we have w · α_{s} ∈ Φ^{−} (otherwise, we have w · Φ^{+} ⊂ Φ^{+} but this is a contradiction). Now the claim follows from Lemma 2.9. (ii) Note that w_{0}(S) ∉ W_{I}, while |G_{I}| ≤ 2 by (i). Thus by Proposion 2.18 (i), N_{W} (W_{I}) is generated by W_{I} and w_{0}(S). Now the claim holds, since w_{0}(S)^{2} = 1. (iii) In this case, we have |Φ^{+} ∖ Φ_{I}| = ∞ by Proposition 2.19. Thus we have G_{I} = 1 by (i), since the set Φ [w] is always finite. Hence the claim holds.

Owing to this description, we have the following:

Corollary 2.21. (i) If W = W (B_{n}), 2 ≤ n < ∞, then ⋂_{i=1}^{n−1} N_{W} (W_{S(B_{i})}) = G_{B_{n}}. (ii) If W = W (D_{n}), 3 ≤ n < ∞, then ⋂_{i=2}^{n−1} N_{W} (W_{S(D_{i})}) = G_{D_{n}} ⋊ ⟨s_{1}⟩.

Proof. Note that, by Lemma 2.12, G_{B_{n}} is generated by all w_{0}(S(B_{k})) (1 ≤ k ≤ n). On the other hand, by Lemma 2.12 again, the product G_{D_{n}}⟨s_{1}⟩ is a semidirect product with G_{D_{n}} normal, and it is generated by all w_{0}(S(D_{k})) (1 ≤ k ≤ n).

We prove the two claims in parallel. Let T = B and L = 1 (for (i)), T = D and L = 2 (for (ii)), respectively. By the above remark, it is enough to show that the group in the left side is generated by all w_{0}(S(T_{k})) (1 ≤ k ≤ n). We use induction on n. First, note that w_{0}(S(T_{n})) ∈ N_{W} (W_{S(T_{i})}) for all L ≤ i ≤ n − 1. Put W′ = W_{S(T_{n−1})}. Then by Corollary 2.20 (ii), we have N_{W} (W′) = W′ ⋊ ⟨w_{0}(S(T_{n}))⟩. Thus the claim holds if n = L + 1; in fact, in this case, W′ = W_{S(T_{L})} is generated by all w_{0}(S(T_{i})) (1 ≤ i ≤ L).

If n > L + 1, then the above equality implies that

\begin{aligned}⋂_{i=L}^{n−1} N_{W} (W_{S(T_{i})}) & = ( ⋂_{i=L}^{n−2} N_{W} (W_{S(T_{i})})) ∩ (W′ ⋊ ⟨w_{0}(S(T_{n}))⟩) \\ & = ( ⋂_{i=L}^{n−2} N_{W′}(W_{S(T_{i})})) ⋊ ⟨w_{0}(S(T_{n}))⟩ \end{aligned}

since w_{0}(S(T_{n})) ∈ ⋂_{i=L}^{n−2} N_{W} (W_{S(T_{i})}). By the induction, the first factor of the semidirect product is generated by all w_{0}(S(T_{i})) (1 ≤ i ≤ n − 1). Thus the claim also holds in this case. Hence the proof is concluded.

We summarize some more properties of the normalizers. First, we have:

If I, J ⊂ S, then N_{W} (W_{I}) ∩ N_{W} (W_{J}) ⊂ N_{W} (W_{I∩J}).  (2.15)
For I ⊂ S, w ∈ N_{W} (W_{I}) if and only if w · Φ_{I} = Φ_{I}.  (2.16)

((2.15) follows from the well-known fact W_{I} ∩ W_{J} = W_{I∩J}. (2.16) follows immediately from (2.11).) Moreover, we have the following:

Lemma 2.22. Let I ⊂ J ⊂ S such that J ∖ I ⊂ I^{⊥}. Then

N_{W} (W_{J}) ∩ N_{W} (W_{I}) ⊂ N_{W} (W_{J∖I}).

Proof. Let w ∈ N_{W} (W_{J}) ∩ N_{W} (W_{I}) and s ∈ J ∖ I. Then w · Φ_{J} = Φ_{J} and w · Φ_{I} = Φ_{I} by (2.16), so that we have w · α_{s} ∈ Φ_{J} and w · α_{s} ∉ Φ_{I}. Now by the hypothesis and (2.10), we have supp(w · α_{s}) ⊂ J ∖ I and so w · α_{s} ∈ Φ_{J∖I}. Hence the claim follows from (2.16).

3 Main results

3.1 Direct indecomposability

In this subsection, we give the main result of this paper; all infinite irreducible Coxeter groups are in fact directly indecomposable, even if it has infinite rank (Theorem 3.3). As is mentioned in Introduction, this result was already shown in [9] for the case of finite rank, in which the finiteness of the ranks is essential and so cannot be removed immediately.

Our proof is based on the following complete description (proved in later sections) of the centralizers of normal subgroups, which are generated by involutions, in irreducible Coxeter groups (possibly of infinite rank):

Theorem 3.1. (See Definition 2.8 for notations.) Let (W, S) be an irreducible Coxeter system of an arbitrary rank, and H ◁ W a normal subgroup generated by involutions. Then:

(i) If H ⊂ Z(W), then Z_{W} (H) = W. (ii) If (W, S) = (W (B_{n}), S(B_{n})), 2 ≤ n ≤ ∞, τ ∈ Aut(Γ (B_{n})), H ⊄ Z(W) and H ⊂ τ(G_{B_{n}}), then Z_{W} (H) = τ(G_{B_{n}}). (cf. Lemma 2.12 for definition of G_{B_{n}}.) (iii) If (W, S) = (W (D_{n}), S(D_{n})), 3 ≤ n ≤ ∞, τ ∈ Aut(Γ (D_{n})), H ⊄ Z(W) and H ⊂ τ(G_{D_{n}}), then Z_{W} (H) = τ(G_{D_{n}}). (cf. Lemma 2.12 for definition of G_{D_{n}}.) (iv) Otherwise, Z_{W} (H) = Z(W).

This theorem yields the following corollary. A group G is said to be a central product of two subgroups H_{1}, H_{2} if G = H_{1}H_{2} and H_{2} ⊂ Z_{G}(H_{1}) (or equivalently H_{1} ⊂ Z_{G}(H_{2})). Note that H_{1} ∩ H_{2} ⊂ Z(G) in this case.

Corollary 3.2. Let (W, S) be an irreducible Coxeter system of an arbitrary rank, and suppose that W is a central product of two subgroups G_{1}, G_{2} generated by involutions. Then either G_{1} ⊂ Z(W) or G_{2} ⊂ Z(W).

Proof. By definition, we have G_{2} ⊂ Z_{W} (G_{1}), W = G_{1}Z_{W} (G_{1}) and G_{1} ◁ W. Now if G_{1} satisfies the condition of cases (ii) or (iii) of Theorem 3.1, then G_{1} and Z_{W} (G_{1}) are contained in the same proper subgroup of W. This is impossible, so that we have G_{1} ⊂ Z(W) (case (i)) or G_{2} ⊂ Z_{W} (G_{1}) = Z(W) (case (iv)).

Now our main result follows immediately:

Theorem 3.3. The only nontrivial direct product decompositions of an irreducible Coxeter group W (of an arbitrary rank) are the ones given in Theorem 2.16. In particular, W is directly indecomposable if and only if W ≄ W (T ) for T = B_{2k+1}, I_{2}(4k + 2) (k ≥ 1), E_{7}, H_{3}.

Proof. Assume that W = G_{1} × G_{2} for nontrivial subgroups G_{1}, G_{2} ⊂ W. Then both G_{1} and G_{2} are generated by involutions, since W is so. Thus by Corollary 3.2, we have either G_{1} = Z(W) or G_{2} = Z(W) (since G_{1}, G_{2} ≠ 1 and |Z(W)| ≤ 2). Hence Z(W) ≠ 1 and so the claim follows from Theorem 2.16.

3.2 The Isomorphism Problem

By using these results, we give some results on the Isomorphism Problem of general Coxeter groups. Let (W, S) be a Coxeter system with canonical direct product decomposition W = ∏_{ω∈Ω} W_{ω} into irreducible components W_{ω}. Then we put

Ω_{fin} = {ω ∈ Ω | |W_{ω}| < ∞}, Ω_{inf} = Ω∖Ω_{fin}, W_{fin} = ∏_{ω∈Ω_{fin}} W_{ω}, W_{inf} = ∏_{ω∈Ω_{inf}} W_{ω}.

(Note that W = W_{fin} ×W_{inf}.) Moreover, we write Ω_{T} = {ω ∈ Ω | W_{ω} ≃ W (T )} for any type T . Now our result (proved later) is stated as follows:

Theorem 3.4. (See above for notations.) Let (W, S), (W′, S′) be two Coxeter systems with the decompositions W = ∏_{ω∈Ω} W_{ω}, W′ = ∏_{ω′∈Ω′} W′_{ω′} into irreducible components. Let π_{ω} : W → W_{ω}, π′_{ω′} : W′ → W′_{ω′} denote the projections.

(i) W ≃ W′ if and only if the following two conditions are satisfied:

(I) There is a bijection φ : Ω_{inf} → Ω′_{inf} such that W_{ω} ≃ W′_{φ(ω)} for all ω ∈ Ω_{inf}.

(II) Each of the following subsets of Ω has the same cardinality as the corresponding subset of Ω′:

Ω_{A_{1}} ∪ ( ⋃_{k≥1} Ω_{B_{2k+1}}) ∪ Ω_{E_{7}} ∪ Ω_{H_{3}} ∪ ( ⋃_{k≥1} Ω_{I_{2}(4k+2)}), Ω_{B_{3}} ∪ Ω_{A_{3}},

Ω_{B_{2k+1}} ∪ Ω_{D_{2k+1}}, Ω_{I_{2}(6)} ∪ Ω_{A_{2}}, Ω_{I_{2}(4k+2)} ∪ Ω_{I_{2}(2k+1)} (k ≥ 2), Ω_{T} for T = A_{n} (4 ≤ n < ∞), B_{n} (n < ∞ even), D_{n} (4 ≤ n < ∞ even),

E_{6}, E_{7}, E_{8}, F_{4}, H_{3}, H_{4}, I_{2}(4k) (2 ≤ k < ∞).

(ii) Suppose that W ≃ W′, and let f ∈ Isom(W, W′). Then:

(I) f(W_{fin}) = W′_{fin} (and so the map g_{fin} defined by g_{fin} = f|_{W_{fin}} is an isomorphism W_{fin} → W′_{fin}).

(II) There is a bijection φ : Ω_{inf} → Ω′_{inf} such that for all ω ∈ Ω_{inf}, the map g_{ω} = π′_{φ(ω)} ∘ f|_{W_{ω}} is an isomorphism W_{ω} → W′_{φ(ω)}.

(III) Moreover, there is a map g_{Z} ∈ Hom(W_{inf}, Z(W′)) such that

f(w) = \begin{cases}g_{ω}(w)g_{Z}(w) & if ω ∈ Ω_{inf}, w ∈ W_{ω}, \\ g_{fin}(w) & if w ∈ W_{fin}. \end{cases}

Note that this is an analogue of the Krull-Remak-Schmidt Theorem on direct product decompositions of groups, and follows from that (together with Theorem 3.3) if W has a composition series. (More precisely, the key property in the proof of the K-R-S Theorem, which follows from the existence of composition series, is that any surjective normal endomorphism of an indecomposable factor is either nilpotent or isomorphic. However, it is not clear whether or not an irreducible Coxeter group has this property.) Our result here is also a generalization of a result of [9].

In order to prove this theorem, we introduce the following “modified version” of irreducible components. Here a group G is said to be admissible if either G is a nontrivial directly indecomposable irreducible Coxeter group (cf. Theorem 3.3) or G is isomorphic to one of W (E_{7})^{+}, W (H_{3})^{+}.

Remark 3.5. Let W = ∏_{ω∈Ω} W_{ω} be the usual decomposition of a Coxeter group W into irreducible components. Then, by subdividing every directly decomposable W_{ω} into the direct factors (cf. Theorem 3.3), we can obtain another decomposition W = ∏_{λ∈Λ} G_{λ} into admissible subgroups G_{λ}. Moreover, since any infinite W_{ω} is directly indecomposable, we can take the index set Λ so that Ω_{inf} ⊂ Λ and G_{ω} = W_{ω} for all ω ∈ Ω_{inf}.

From now, we consider the following two conditions on a family G of groups:

If {G_{λ}}_{λ∈Λ} is a subfamily of G, G′ ∈ G and f : G = ∏_{λ∈Λ} G_{λ} → G′ is a

surjective homomorphism, then f maps a G_{λ} onto G′ (so that it maps all other G_{μ} into Z(G′)).

(3.1)

If G ∈ G, then either Z(G) = 1 or Z(G) is cyclic of prime order. (3.2)

(Actually, the condition (3.2) can be slightly weakened to the form that Z(G) is either trivial or a finite elementary abelian p-group with p prime. But we omit the detail here, since we do not need such a generalization in this paper.)

Remark 3.6. (i) If G satisfies (3.1), then all groups G ∈ G are directly indecomposable. In fact, if G admits a nontrivial decomposition G = G_{1} × G_{2} with projections π_{i} : G → G_{i} (i = 1, 2), then the map G × G → G, (w, u) ↦ π_{1}(w)π_{2}(u) is surjective but does not satisfy the conclusion of (3.1). (ii) If G satisfies (3.1) and (3.2), then any G ∈ G has the three properties (I)–(III) in Lemma 2.1 whenever Z(G) ≠ G. This follows immediately from (i).

Lemma 3.7. Any family G of admissible groups satisfies (3.1) and (3.2).

Proof. The condition (3.2) follows from Lemma 2.17. For (3.1), we may assume G′ ≄ W (A_{1}) (so that Z(G′) ≠ G′), since otherwise the conclusion is obvious. Then there is an index λ ∈ Λ such that f(G_{λ}) ⊄ Z(G′). Put G_{1} = G_{λ} and G_{2} = ∏_{μ∈Λ∖{λ}} G_{μ}. Then the hypothesis of (3.1) implies that G′ is a central product (cf. Section 3.1) of f(G_{1}) and f(G_{2}), so that f(G_{1}) ∩ f(G_{2}) ⊂ Z(G′). Thus the conclusion follows from Lemma 2.17 if G′ ≃ W (E_{7})^{+} or W (H_{3})^{+} (in fact, the central product is a direct product since Z(G′) = 1, while G′ is directly indecomposable).

On the other hand, suppose that G′ is a directly indecomposable irreducible Coxeter group. Since both G_{1} and G_{2} are generated by involutions (cf. Lemma 2.17), f(G_{1}) and f(G_{2}) also have this property. Thus we have f(G_{2}) ⊂ Z(G′) by Corollary 3.2 (since f(G_{1}) ⊄ Z(G′)). Now if Z(G′) ⊄ f(G_{1}) (so that f(G_{1})∩ Z(G′) = 1 since |Z(G′)| ≤ 2), then the central product becomes a (nontrivial) direct product, but this is impossible. This implies that f(G_{2}) ⊂ Z(G′) ⊂ f(G_{1}) and so f(G_{1}) = G′. Hence the claim holds.

Remark 3.8. By a similar argument, it is deduced that any family G, consisting of cyclic groups of prime order and directly indecomposable groups with trivial center, also satisfies the conditions (3.1) and (3.2).

We prepare some more notations. For a decomposition G = ∏_{λ∈Λ} G_{λ} of G, put

G_{Λ′} = ∏_{λ∈Λ′} G_{λ} (for Λ′ ⊂ Λ), Λ_{Z} = {λ | Z(G_{λ}) = G_{λ}}, Λ_{¬Z} = Λ ∖ Λ_{Z},

Λ_{p} = {λ | |Z(G_{λ})| = p}, Λ_{Z,p} = Λ_{Z} ∩ Λ_{p}, Λ_{¬Z,p} = Λ_{¬Z} ∩ Λ_{p} (p prime or 1). (3.3)

Note that the proof of the following theorem is essentially the same as the proof of Theorem 2.1 of [9], but slightly more delicate by the lack of the assumption on finiteness of the index sets (not only by generality of the context). Note also that this is also an analogue of the Krull-Remak-Schmidt Theorem.

Theorem 3.9. (See (3.3) for notations.) Let G = ∏_{λ∈Λ} G_{λ}, G′ = ∏_{λ′∈Λ′} G′_{λ′} be decompositions of two groups G, G′ into nontrivial subgroups. Let π_{λ} : G → G_{λ} and π′_{λ′} : G′ → G′_{λ′} be the projections. Suppose that G = {G_{λ} | λ ∈ Λ} ∪ {G′_{λ′} | λ′ ∈ Λ′} satisfies the conditions (3.1) and (3.2). Let f ∈ Isom(G, G′). Then:

(i) There is a bijection φ : Λ → Λ′ such that G_{λ} ≃ G′_{φ(λ)} for all λ ∈ Λ. Moreover, for any λ ∈ Λ_{¬Z}, the map g_{λ} = π′_{φ(λ)} ∘ f|_{G_{λ}} is an isomorphism G_{λ} → G′_{φ(λ)}. (ii) Moreover, there is a map g_{Z} ∈ Hom(G, Z(G′)) such that

f(w) = \begin{cases}g_{λ}(w)g_{Z}(w) & if λ ∈ Λ_{¬Z}, w ∈ G_{λ}, \\ g_{Z}(w) & if w ∈ G_{Λ_{Z}} \end{cases}

and that π′_{φ(λ)} ∘ g_{Z}(G_{λ}) = 1 for all λ ∈ Λ_{¬Z}. (iii) If ⋃_{p≠1} Λ_{p} ⊂ Λ^{♮} ⊂ Λ, then ⋃_{p≠1} Λ′_{p} ⊂ φ(Λ^{♮}) and f(G_{Λ^{♮}}) = G′_{φ(Λ^{♮})}.

Proof. Note that ⋃_{p≠1} Λ_{p} = {λ ∈ Λ | Z(G_{λ}) ≠ 1}. Then the claim (iii) is deduced from the other claims (since now Z(G) ⊂ G_{Λ^{♮}} and Z(G′) ⊂ G′_{φ(Λ^{♮})}).

From now, we prove the claims (i) and (ii). First, we put (symmetrically)

f_{λ′} = π′_{λ′} ∘ f ∈ Hom(G, G′_{λ′}) (λ′ ∈ Λ′), f′_{λ} = π_{λ} ∘ f^{−1} ∈ Hom(G′, G_{λ}) (λ ∈ Λ),

and define (symmetrically)

A′_{λ} = {λ′ ∈ Λ′ | f_{λ′}(G_{λ}) ⊄ Z(G′_{λ′})} ⊂ Λ′_{¬Z} for λ ∈ Λ_{¬Z},
A_{λ′} = {λ ∈ Λ | f′_{λ}(G′_{λ′}) ⊄ Z(G_{λ})} ⊂ Λ_{¬Z} for λ′ ∈ Λ′_{¬Z}.

Note that A′_{λ} ≠ ∅ since f(G_{λ}) ⊄ Z(G′) (and A_{λ′} ≠ ∅ by symmetry). Moreover, since f_{λ′} : G → G′_{λ′} is surjective, the condition (3.1) implies that

if λ′ ∈ A′_{λ}, then f_{λ′}(G_{λ}) = G′_{λ′} and f_{λ′}(G_{μ}) ⊂ Z(G′_{λ′}) for all μ ∈ Λ ∖ {λ}.

By symmetry, a similar property holds for λ ∈ A_{λ′} (with respect to the map f′_{λ}).

We prove the following claims:

Claim 1: If λ, μ ∈ Λ_{¬Z} and λ ≠ μ, then A′_{λ} ∩ A′_{μ} = ∅.

Claim 2: If λ′ ∈ A′_{λ}, then λ ∈ A_{λ′}. (Thus |A′_{λ}| = 1 for all λ ∈ Λ_{¬Z}, by Claim 1 and symmetry. Moreover, by symmetry, the map φ : Λ_{¬Z} → Λ′_{¬Z} defined by A′_{λ} = {φ(λ)} is a bijection with inverse map satisfying A_{λ′} = {φ^{−1}(λ′)}.)

Claim 3: The map g_{λ} (λ ∈ Λ_{¬Z}) in (i) is an isomorphism G_{λ} → G′_{φ(λ)}. Claim 4: f(Z(G_{Λ_{¬Z,p}})) = Z(G′_{Λ′_{¬Z,p}}) for all primes p. Claim 5: For each prime p, Λ_{Z,p} and Λ′_{Z,p} have the same cardinality. Proof of Claim 1: Assume contrary that λ′ ∈ A′_{λ} ∩ A′_{μ}. Then the relation

λ′ ∈ A′_{λ} means that f_{λ′}(G_{λ}) ⊄ Z(G′_{λ′}), while the relation λ′ ∈ A′_{μ} implies (by the above property) that f_{λ′}(G_{λ}) ⊂ Z(G′_{λ′}) (since λ ≠ μ). This is a contradiction.

Proof of Claim 2: Since G′_{λ′} ≠ Z(G′_{λ′}), we can take an element w ∈ G′_{λ′} ∖ Z(G′_{λ′}). Put u_{μ} = f′_{μ}(w) ∈ G_{μ} for μ ∈ Λ, so that we have w = f(∏_{μ∈Λ} u_{μ}). It suffices to show u_{λ} = f′_{λ}(w) ∉ Z(G_{λ}). Now f_{λ′}(u_{μ}) ∈ Z(G′_{λ′}) for all μ ∈ Λ∖{λ}, while w = π′_{λ′}(w) ∉ Z(G′_{λ′}). Thus we have f_{λ′}(u_{λ}) ∉ Z(G′_{λ′}) and so u_{λ} ∉ Z(G_{λ}) (since f_{λ′}(G_{λ}) = G′_{λ′}). Hence λ ∈ A_{λ′}.

Proof of Claim 3: Note that g_{λ} : G_{λ} → G′_{φ(λ)} is surjective (as above). Now the following equivalence holds for all w ∈ G_{λ}:

f_{φ(λ)}(w) ∈ Z(G′_{φ(λ)}) ⟺ f(w) ∈ Z(G′) ⟺ w ∈ Z(G) ⟺ w ∈ Z(G_{λ})

(we use the fact A′_{λ} = {φ(λ)} for the first equivalence). This implies that ker g_{λ} is contained in the simple group Z(G_{λ}) (cf. (3.2)), so that ker g_{λ} = 1 or Z(G_{λ}). Thus g_{λ} is injective (and so an isomorphism) if Z(G_{λ}) = 1. Moreover, if Z(G′_{φ(λ)}) = 1, then f′_{λ}|_{G′_{φ(λ)}} is an isomorphism G′_{φ(λ)} → G_{λ} by symmetry, so that we have Z(G_{λ}) = 1. Thus g_{λ} is injective (as above) also in this case.

On the other hand, suppose Z(G′_{φ(λ)}) ≠ 1. Then by the above equivalence, there is an element w ∈ Z(G_{λ}) such that g_{λ}(w) ≠ 1 (since g_{λ} is surjective). Thus we have ker g_{λ} ≠ Z(G_{λ}) and so ker g_{λ} = 1. Hence g_{λ} is an isomorphism.

Proof of Claim 4: Note that Z(G) = ∏_{p≠1} Z(G_{Λ_{p}}) and each Z(G_{Λ_{p}}) is an elementary abelian p-group, by (3.2). Z(G′) also admits a similar decomposition. Thus the isomorphism f|_{Z(G)} : Z(G) → Z(G′) maps each Z(G_{Λ_{p}}) onto

Z(G′_{Λ′_{p}}). Moreover, for any λ ∈ Λ_{¬Z,p}, the composite homomorphism G_{λ} →^{f} G′ → G′_{Λ′_{Z,p}} (where the latter map is the projection) maps Z(G_{λ}) to 1, by Remark 3.6 (ii) (note that Z(G′_{Λ′_{Z,p}}) = G′_{Λ′_{Z,p}}). Thus we have f(Z(G_{λ})) ⊂ G′_{Λ′_{¬Z,p}} for any λ ∈ Λ_{¬Z,p} and so f(Z(G_{Λ_{¬Z,p}})) ⊂ Z(G′_{Λ′_{¬Z,p}}). Now this claim holds by symmetry.

Proof of Claim 5: Note that Z(G_{Λ_{p}}) = G_{Λ_{Z,p}} × Z(G_{Λ_{¬Z,p}}) and Z(G′_{Λ′_{p}}) admits a similar decomposition. Moreover, we have f(Z(G_{Λ_{p}})) = Z(G′_{Λ′_{p}}) and f(Z(G_{Λ_{¬Z,p}})) = Z(G′_{Λ′_{¬Z,p}}) by Claim 4. Thus the complementary factors G_{Λ_{Z,p}}, G′_{Λ′_{Z,p}}, which are elementary abelian p-groups with basis having the same cardinality as Λ_{Z,p}, Λ′_{Z,p} respectively, are also isomorphic. Now this claim follows from uniqueness of the dimension of a vector space.

Conclusion. Since Λ_{Z}, Λ′_{Z} are disjoint unions of Λ_{Z,p}, Λ′_{Z,p} respectively (cf. (3.2)), Claim 5 implies that this φ extends (not uniquely) to a bijection φ : Λ → Λ′ satisfying (i) (note that Λ_{Z,1} = Λ′_{Z,1} = ∅ by the hypothesis). Moreover, define a map g_{Z} : G → Z(G′) componentwise by

g_{Z}(w) = \begin{cases}∏_{λ′∈Λ′∖\{φ(λ)\}} f_{λ′}(w) & if λ ∈ Λ_{¬Z}, w ∈ G_{λ}, \\ f(w) & if w ∈ G_{Λ_{Z}} . \end{cases}

Note that G_{Λ_{Z}} ⊂ Z(G), while in the above definition, we have f_{λ′}(w) ∈ Z(G′_{λ′}) by the fact A′_{λ} = {φ(λ)}. Since Z(G′) is abelian, these facts imply that g_{Z} is a well-defined group homomorphism. Now the claim (ii) follows from definition.

Proof of Theorem 3.4. Let W = ∏_{λ∈Λ} G_{λ}, W′ = ∏_{λ′∈Λ′} G′_{λ′} be the decompositions into admissible groups given in Remark 3.5.

(i) Each of the sets in the condition (II), except Ω_{E_{7}} and Ω_{H_{3}} in the last row, has the same cardinality as the set {λ ∈ Λ | G_{λ} ≃ W (T′)} where T′ = A_{1}, A_{3}, D_{2k+1}, A_{2}, I_{2}(2k + 1) and T , respectively (note that no two admissible finite groups of distinct types are isomorphic; cf. Lemma 2.17). Moreover, each of Ω_{E_{7}} and Ω_{H_{3}} has the same cardinality as {λ ∈ Λ | G_{λ} ≃ W (T′)^{+}} for T′ = E_{7} and H_{3}, respectively. Similar relations also hold for W′. Thus the two conditions (I), (II) are satisfied if and only if there is a bijection ψ : Λ → Λ′ such that G_{λ} ≃ G′_{ψ(λ)} for all λ ∈ Λ. Hence the claim follows from Theorem 3.9 (i) (which can be applied indeed to the case, by Lemma 3.7). (ii) Take φ : Λ → Λ′, g_{λ} ∈ Isom(G_{λ}, G′_{φ(λ)}) (λ ∈ Λ_{¬Z}) and g′_{Z} ∈ Hom(W, Z(W′)) as in the conclusion of Theorem 3.9. By Remark 3.5, g_{ω} ∈ Isom(W_{ω}, W′_{φ(ω)}) for all ω ∈ Ω_{inf}, so that the claim (II) holds. The claim (I) follows from Theorem 3.9 (iii) (by putting Λ^{♮} = Λ ∖ Ω_{inf}). Moreover, the claim (III) also follows from Theorem 3.9, by putting g_{Z} = g′_{Z}|_{W_{inf}} . Hence the proof is concluded.

3.3 Automorphism groups

In this subsection, the complete direct product of groups is denoted by a symbol \overline{∏}. Owing to Theorems 3.4 and 3.9, we can examine the automorphism groups of W = ∏_{ω∈Ω} W_{ω} and G = ∏_{λ∈Λ} G_{λ} respectively (Theorem 3.10), under the notations and hypotheses in Section 3.2. Note that each Aut(G_{λ}), Aut(W_{ω}) is embedded into Aut(G), Aut(W) respectively. The group Aut(W_{fin}) is also embedded into Aut(W).

On the other hand, the symmetric group on each isomorphism class of components of G or W is also embedded into the automorphism group, as follows. For the case of G, we partition the index set Λ_{¬Z} into subsets Λ_{ξ} (ξ ∈ Ξ) so that λ, λ′ ∈ Λ_{¬Z} are in the same subset if and only if G_{λ} ≃ G_{λ′}. Moreover, for ξ ∈ Ξ, we choose an “identity map” id_{μ,λ} ∈ Isom(G_{λ}, G_{μ}) for each λ, μ ∈ Λ_{ξ} so that id_{λ,λ} = id_{G_{λ}}, id_{λ,μ} = id_{μ,λ}^{−1} and id_{ν,μ} ∘ id_{μ,λ} = id_{ν,λ} for all λ, μ, ν ∈ Λ_{ξ}. (This can be done by taking a maximal tree in the category of groups G_{λ} (λ ∈ Λ_{ξ}) and group isomorphisms.) Then each element τ of the symmetric group Sym(Λ_{ξ}) on Λ_{ξ} induces an automorphism of the factor G_{Λ_{ξ}} of G; namely,

τ(w) = id_{τ(λ),λ}(w) ∈ G_{τ(λ)} for λ ∈ Λ_{ξ} and w ∈ G_{λ}.

In this manner, Sym(Λ_{ξ}) is embedded into Aut(G_{Λ_{ξ}}), and so also into Aut(G). Similarly, we write Ω = ⨆_{υ∈Υ} Ω_{υ}, choose “identity maps” id_{ω′,ω} ∈ Isom(W_{ω}, W_{ω′}) and then embed every symmetric group Sym(Ω_{υ}) into Aut(W). Moreover, put

Υ_{fin} = {υ ∈ Υ | |W_{ω}| < ∞ for ω ∈ Ω_{υ}} and Υ_{inf} = Υ ∖ Υ_{fin}.

Recall (Section 2.1) the structure of the monoid Hom(G′, Z(G′)) (where G′ is a group), the action of Aut(G′) on it and the embedding f ↦ f^{♭} into the monoid End(G′) compatible with the action of Aut(G′). For a subset H ⊂ Hom(G′, Z(G′)), the image of H by the embedding is denoted by H^{♭}. In particular, the group Hom(G′, Z(G′))^{×} of invertible elements of Hom(G′, Z(G′)) is embedded into Aut(G′) (as a subgroup Hom(G′, Z(G′))^{×}^{♭}).

Now for the group G, let

Hom(G, Z(G))_{o} = {f ∈ Hom(G, Z(G)) |f(G_{Λ_{Z}}) = 1,
f(G_{λ}) ⊂ Z(G_{λ}) for all λ ∈ Λ_{¬Z}}

(cf. (3.3) for notations). Since we assumed that each G_{λ} (λ ∈ Λ_{¬Z}) satisfies the three conditions in Lemma 2.1 (cf. Remark 3.6 (ii)), we have f(Z(G)) = 1 for all f ∈ Hom(G, Z(G))_{o}. Thus by Lemma 2.4 (i), Hom(G, Z(G))_{o} is an abelian subgroup of Hom(G, Z(G))^{×} with multiplication (f ∗ g)(w) = f(w)g(w) (f, g ∈ Hom(G, Z(G))_{o}, w ∈ G).

On the other hand, since Z(W_{inf}) = 1, Lemma 2.4 (ii) implies that the set Hom(W_{inf}, Z(W)) forms an abelian normal subgroup of Hom(W, Z(W))^{×} with multiplication (f ∗ g)(w) = f(w)g(w) (f, g ∈ Hom(W_{inf}, Z(W)), w ∈ W_{inf}). Since now Z(W) is an elementary abelian 2-group, Hom(W_{inf}, Z(W)) is also an elementary abelian 2-group.

Now our result is stated as follows. Here G′_{1}G′_{2} denotes (for two subgroups G′_{1}, G′_{2} of a group G′) the subgroup of G′ generated by G′_{1} ∪ G′_{2}, as usual.

Theorem 3.10. (See above for notations. See also Section 3.2.)

(i) Put H_{1} = Hom(G, Z(G))^{×}^{♭}, H_{2} = \overline{∏}_{λ∈Λ_{¬Z}} Aut(G_{λ}), H_{3} = \overline{∏}_{ξ∈Ξ}Sym(Λ_{ξ}) and H_{4} = Hom(G, Z(G))_{o}^{♭}. Then

Aut(G) = (H_{1}H_{2}) ⋊ H_{3}, H_{1} ◁ Aut(G), H_{2} ◁ H_{2}H_{3}, H_{1} ∩ H_{2} = H_{4}.

(ii) Put H′_{1} = Hom(W_{inf}, Z(W))^{♭}, H′_{2} = Aut(W_{fin}), H′_{3} = \overline{∏}_{ω∈Ω_{inf}} Aut(W_{ω}) and H′_{4} = \overline{∏}_{υ∈Υ_{inf}} Sym(Ω_{υ}). Then

Aut(W) = (H′_{1} ⋊ (H′_{2} × H′_{3})) ⋊ H′_{4}, H′_{2}H′_{4} = H′_{2} × H′_{4}, H′_{3}H′_{4} = H′_{3} ⋊ H′_{4}.

(iii) The subgroup H = (\overline{∏}_{ω∈Ω}Aut(W_{ω})) (\overline{∏}_{υ∈Υ}Sym(Ω_{υ})) has finite index in Aut(W) if and only if, either Z(W) = 1 or the odd Coxeter graph (cf. Definition 2.10) Γ^{odd} of W consists of only finitely many connected components. (Hence the index is finite whenever W has finite rank.)

From now, we prove this theorem. First, we prove (i) and (ii). Note that H′_{2}H′_{3} = H′_{2} × H′_{3} and H′_{2}H′_{4} = H′_{2} × H′_{4} by definition. Moreover, by definition,

H_{2} = {f ∈ Aut(G) | f(w) = w (w ∈ G_{Λ_{Z}}), f(G_{λ}) = G_{λ} (λ ∈ Λ_{¬Z})}, H′_{3} = {f ∈ Aut(W) | f(w) = w (w ∈ W_{fin}), f(W_{ω}) = W_{ω} (ω ∈ Ω_{inf})}.

(3.4)

Claim 1. (i) Aut(G) = H_{1}H_{2}H_{3}. (ii) Aut(W) = H′_{1}H′_{2}H′_{3}H′_{4}.

Proof. (i) Let f ∈ Aut(G), and take φ, g_{λ}, g_{Z} as in Theorem 3.9. Note that φ(Λ_{ξ}) = Λ_{ξ} for all ξ ∈ Ξ. Now define f_{1} ∈ Hom(G, Z(G)) by

f_{1}(w) = \begin{cases}g_{Z} ∘ g_{φ^{−1}(λ)}^{−1}(w)^{−1} & for λ ∈ Λ_{¬Z}, w ∈ G_{λ}, \\ wf(w)^{−1} & for w ∈ G_{Λ_{Z}} \end{cases}

(this is well defined since G_{Λ_{Z}} ⊂ Z(G)). Then by definition and Theorem 3.9, we have f = f_{1}^{♭} ∘ f_{2} ∘ f_{3}, where

f_{2} = (g_{φ^{−1}(λ)} ∘ id_{φ^{−1}(λ),λ})_{λ∈Λ_{¬Z}} ∈ H_{2}, f_{3} = (φ|_{Λ_{ξ}})_{ξ∈Ξ} ∈ H_{3}.

Moreover, we have f_{1}^{♭} = f ∘ f_{3}^{−1} ∘ f_{2}^{−1} ∈ Aut(G) and so f_{1} ∈ Hom(G, Z(G))^{×} by Lemma 2.2 (ii). Hence f_{1}^{♭} ∈ H_{1} and so f ∈ H_{1}H_{2}H_{3}. (ii) Let f ∈ Aut(W), and take φ, g_{fin}, g_{λ}, g_{Z} as in Theorem 3.4 (ii). Note that φ(Ω_{υ}) = Ω_{υ} for all υ ∈ Υ. Now define f_{1} ∈ Hom(W_{inf}, Z(W)) by

f_{1}(w) = g_{Z} ∘ g_{φ^{−1}(ω)}^{−1}(w)^{−1} for ω ∈ Ω_{inf}, w ∈ W_{ω}.

Then we have (by definition and Theorem 3.4 (ii))

f = f_{1}^{♭} ∘ g_{fin} ∘ (g_{φ^{−1}(ω)} ∘ id_{φ^{−1}(ω),ω})_{ω∈Ω_{inf}} ∘ (φ|_{Ω_{υ}})_{υ∈Υ_{inf}} ∈ H′_{1}H′_{2}H′_{3}H′_{4}.

Hence the proof is concluded.

Claim 2. (i) If f^{♭} ∈ H_{1}, λ, μ ∈ Λ_{¬Z} and f^{♭}(G_{λ}) ⊂ G_{μ}, then λ = μ and f(G_{λ}) ⊂ Z(G_{λ}). (ii) If f^{♭} ∈ H′_{1}, ω, ω′ ∈ Ω_{inf} and f^{♭}(W_{ω}) ⊂ W_{ω′}, then ω = ω′ and f(W_{ω}) = 1.

Proof. (i) By the choice of λ, we can take w ∈ G_{λ} ∖ Z(G_{λ}). Now we have π_{λ}(f(w)) ∈ Z(G_{λ}) (where π_{λ} is the projection G → G_{λ}) and so π_{λ}(f^{♭}(w)) = wπ_{λ}(f(w))^{−1} ≠ 1. Since f^{♭}(w) ∈ G_{μ}, this implies that μ = λ. Now the latter part follows from definition of the map f^{♭}. (ii) By a similar argument to (i), we have ω = ω′ and f(W_{ω}) ⊂ Z(W_{ω}). Hence the claim holds since Z(W_{ω}) = 1.

Claim 3. (i) (H_{1}H_{2}) ∩ H_{3} = 1. (ii) (H′_{1}H′_{2}H′_{3}) ∩ H′_{4} = 1.

Proof. (i) Let f_{1} ∈ H_{1}, f_{2} ∈ H_{2} such that f_{1}∘f_{2} ∈ H_{3}. By (3.4) and definition of H_{3}, both f_{2}^{−1} and f_{1} ∘f_{2} map each component G_{λ} (λ ∈ Λ_{¬Z}) onto a component, so that f_{1} also does so. By Claim 2 (i), f_{1} maps each G_{λ} (λ ∈ Λ_{¬Z}) onto itself, while f_{2} also does so (cf. (3.4)). Thus f_{1} ∘ f_{2} ∈ H_{3} also has this property. By definition of H_{3}, this occurs only if f_{1} ∘ f_{2} = id_{G}. Hence the claim holds. (ii) The proof is similar to (i); if f_{i} ∈ H′_{i} (i = 1, 2, 3) and f_{4} = f_{1} ∘ f_{2} ∘ f_{3} ∈ H′_{4}, then f_{1} = f_{4} ∘ f_{3}^{−1} ∘ f_{2}^{−1} must map each W_{ω} (ω ∈ Ω_{inf}) onto some component, which is W_{ω} by Claim 2 (ii). This implies that f_{4} maps each W_{ω} (ω ∈ Ω_{inf}) onto itself, so that f_{4} = id_{W} by definition of H′_{4}. Hence the claim holds.

Claim 4. (i) H_{2} ◁ H_{2}H_{3}. (ii) H′_{3} ◁ H′_{3}H′_{4}.

Proof. For (i), it is enough to show that f_{3} ∘ f_{2} ∘ f_{3}^{−1} ∈ H_{2} for all f_{2} ∈ H_{2} and f_{3} ∈ H_{3}. By definition, f_{3} is identity on G_{Λ_{Z}} and maps each G_{λ} (λ ∈ Λ_{¬Z}) onto a component. Now by (3.4), f_{3} ∘ f_{2} ∘ f_{3}^{−1} also satisfies the condition in (3.4), so that it belongs to H_{2}. Hence the claim holds. The proof of (ii) is similar.

Claim 5. (i) H_{1} ◁ Aut(G). (ii) H′_{1} ◁ Aut(W).

Proof. (i) Note that Aut(G) acts on the monoid Hom(G, Z(G)). Thus its subgroup Hom(G, Z(G))^{×} of the invertible elements is invariant under the action. Now the claim follows from Lemma 2.2 (iii). (ii) By Lemma 2.2 (iii), it is enough to show that the subgroup Hom(W_{inf}, Z(W)) of Hom(W, Z(W)) is invariant under the action of Aut(W). Moreover, by Claim 1, it is enough to show that h ∘ f ∘ h^{−1} ∈ Hom(W_{inf}, Z(W)) for all f ∈ Hom(W_{inf}, Z(W)) and h ∈ H′_{2}H′_{3}H′_{4}. Now we have h(W_{fin}) = W_{fin} by definition of H′_{2}, H′_{3} and H′_{4}, so that h ∘ f ∘ h^{−1}(W_{fin}) = h(f(W_{fin})) = h(1) = 1. Hence the claim holds.

Claim 6. (i) H_{1} ∩ H_{2} = H_{4}. (ii) H′_{1} ∩ (H′_{2}H′_{3}) = 1.

Proof. (i) Let f^{♭} ∈ H_{1} ∩H_{2}. Then by (3.4), we have f^{♭}(w) = w (or equivalently f(w) = 1) for all w ∈ G_{Λ_{Z}} and f^{♭}(G_{λ}) = G_{λ} for all λ ∈ Λ_{¬Z}. Thus we have f ∈ Hom(G, Z(G))_{o} by Claim 2 (i), so that f^{♭} ∈ H_{4}. Conversely, H_{4} ⊂ H_{1} by definition, while H_{4} ⊂ H_{2} by (3.4) and definition of H_{4}. Hence the claim holds. (ii) Let f^{♭} ∈ H′_{1} ∩ (H′_{2}H′_{3}). Then for any ω ∈ Ω_{inf}, we have f^{♭}(W_{ω}) = W_{ω} by definition of H′_{2} and H′_{3}. Thus we have f(W_{ω}) = 1 by Claim 2 (ii). Hence f = 1 and f^{♭} = id_{W}.

Now the claims (i) and (ii) of Theorem 3.10 hold. Namely:

(i) We have H_{1} ∩H_{2} = H_{4} (Claim 6) and Aut(G) = (H_{1}H_{2})H_{3} (Claim 1). Now since H_{1} ◁ Aut(G) (Claim 5) and H_{2} ◁ H_{2}H_{3} (Claim 4), the conjugation by an element of H_{1}, H_{2} or H_{3} maps H_{1} and H_{2} into the subgroup H_{1}H_{2} generated by H_{1} ∪ H_{2}. Thus H_{1}H_{2} ◁ Aut(G) and Aut(G) = (H_{1}H_{2}) ⋊ H_{3} (Claim 3).

(ii) We have H′_{2}H′_{3} = H′_{2} × H′_{3}, H′_{2}H′_{4} = H′_{2} × H′_{4} (as the above remark), H′_{3}H′_{4} = H′_{3} ⋊ H′_{4} (Claims 3, 4) and H′_{1} ◁ Aut(W) (Claim 5), so H′_{1}H′_{2}H′_{3} = H′_{1} ⋊ (H′_{2} × H′_{3}) (Claim 6). Now by a similar argument to (i), the conjugation by an element of H′_{4} leaves H′_{1}H′_{2}H′_{3} invariant, so H′_{1}H′_{2}H′_{3} ◁ Aut(W) since Aut(W) = H′_{1}H′_{2}H′_{3}H′_{4} (Claim 1). Thus Aut(W) = (H′_{1} ⋊ (H′_{2} × H′_{3})) ⋊ H′_{4} (Claim 3).

Proof of Theorem 3.10 (iii). If Z(W) = 1, then all irreducible components of W are directly indecomposable (cf. Theorem 3.3), so that the decomposition W = ∏_{ω∈Ω} W_{ω} itself satisfies the conditions (3.1) and (3.2) in Section 3.2. Thus we can apply the result (i) to this decomposition. Now H_{1} = 1 since Z(W) = 1. Moreover, Ω = Ω_{¬Z} in this case, so that we have H = H_{2}H_{3} = Aut(W).

From now, we assume that Z(W) ≠ 1. For f ∈ Aut(W), let sep(f) be the set of all ω ∈ Ω such that f(W_{ω}) ⊄ W_{ω′} for all ω′ ∈ Ω. Since any element of H maps each component W_{ω} onto a component, the cardinality of the set sep(f) is invariant in each coset of Aut(W)/H. Moreover, by definition, we have

H = (\overline{∏}_{ω∈Ω_{fin}}Aut(W_{ω})) (\overline{∏}_{υ∈Υ_{fin}}Sym(Ω_{υ})) × H′_{3}H′_{4} ⊂ H′_{2} × (H′_{3}H′_{4}).

Case 1. Γ^{odd} consists of only finitely many connected components: This implies that |Ω| < ∞ and |Hom(W_{inf}, {±1})| < ∞ (cf. Lemma 2.11). Since Z(W) is now a finite elementary abelian 2-group, (ii) implies that H′_{2}H′_{3}H′_{4} has index |H′_{1}| = |Hom(W_{inf}, Z(W))| < ∞ in Aut(W). Moreover, since now |W_{fin}| < ∞, the index of H in H′_{2}H′_{3}H′_{4} is ≤ |H′_{2}| < ∞. Thus H has finite index also in Aut(W).

Case 2. Γ^{odd} consists of infinitely many connected components: Now we have to show that H has infinite index in Aut(W).

Subcase 2-1. The odd Coxeter graph of some W_{ω} consists of infinitely many connected components: Note that ω ∈ Ω_{inf} in this case. Now by Lemma 2.11, we have |Hom(W_{ω}, {±1})| = ∞ and so |Hom(W_{inf}, Z(W))| = ∞ (since we assumed that Z(W) ≠ 1). Thus by (ii), the subgroup H′_{2}H′_{3}H′_{4} (⊃ H) has index |H′_{1}| = ∞, so that H also has infinite index in Aut(W).

Subcase 2-2. The odd Coxeter graph of every W_{ω} consists of only finitely many connected components: Then we have |Ω| = ∞ by the hypothesis of Case 2. Since we assumed that Z(W) ≠ 1, we can take an infinite sequence ω_{0}, ω_{1}, ω_{2}, … of distinct elements of Ω such that Z(W_{ω_{0}}) ≠ 1. Let u denote the unique element of Z(W_{ω_{0}}) ∖ {1}. Now for k ≥ 1, we define f_{k} ∈ Hom(W, Z(W)) componentwise by

f_{k}(w) = \begin{cases}u^{ℓ(w)} & if ω ∈ \{ω_{1}, …, ω_{k}\} and w ∈ W_{ω}, \\ 1 & if ω ∈ Ω ∖ \{ω_{1}, …, ω_{k}\} and w ∈ W_{ω}. \end{cases}

Then we have f_{k} ∘f_{k} = 1 and so f_{k} ∗f_{k} = 1 since Z(W) is an elementary abelian 2-group. This implies that f_{k} ∈ Hom(W, Z(W))^{×} and so f_{k}^{♭} ∈ Aut(W), while sep(f_{k}^{♭}) = {ω_{1}, …, ω_{k}} by definition. Thus by the above remark, all f_{k}^{♭} belong to distinct cosets in Aut(W)/H and so H has infinite index in Aut(W). Hence the proof is concluded.

Example 3.11. Let m = (m_{1}, m_{2}, …) be an infinite sequence of nonnegative integers. Here we examine Aut(W_{m}) for the group W_{m} = ∏_{n≥1}(Sym_{n})^{m_{n}} by using our result, where Sym_{n} = Sym({1, 2, …, n}) is the symmetric group of degree n. Note that Sym_{1} = 1.

Since Sym_{n} (n ≥ 2) is the Coxeter group W (A_{n−1}), which is directly indecomposable (cf. Theorem 3.3), we can apply Theorem 3.10 (i) to this decomposition of W_{m}. In this case, we have Z(Sym_{n}) = 1 unless Z(Sym_{n}) = Sym_{n} (namely n = 1, 2), so that Hom(W_{m}, Z(W_{m}))_{o} = 1. Thus we have Aut(W_{m}) = H_{1} ⋊ H_{2} ⋊ H_{3}.

Note that Z(W_{m}) = (Sym_{2})^{m_{2}} ≃ {±1}^{m_{2}}, while |Hom(Sym_{n}, {±1})| = 2 for all n ≥ 2 by Lemma 2.11. Thus Lemmas 2.3 and 2.4 (ii) imply that

H_{1} = Hom(∏_{n≥3}(Sym_{n})^{m_{n}}, Z(W_{m}))^{♭} ⋊ Hom(Sym_{2}^{m_{2}}, Z(W_{m}))^{×}^{♭}

= (\overline{∏}_{n≥3}Hom((Sym_{n})^{m_{n}}, Z(W_{m})))^{♭} ⋊ Aut((Sym_{2})^{m_{2}})
≃ (\overline{∏}_{n≥3}\{±1\}^{m_{2}m_{n}}) ⋊ GL_{m_{2}}(𝔽_{2}).

Secondly, recall the well-known fact that Aut(Sym_{n}) = Inn(Sym_{n}) (the group of inner automorphisms) if n ≠ 6 and |Aut(Sym_{6})/Inn(Sym_{6})| = 2. This implies that Aut(Sym_{2}) = 1, |Aut(Sym_{6})| = 2|Sym_{6}| and Aut(Sym_{n}) ≃ Sym_{n} if n ≠ 2, 6. Thus we have

H_{2} ≃ \overline{∏}_{n≥3}Aut(Sym_{n})^{m_{n}} ≃ (\overline{∏}_{3≤n≠6}Sym_{n}^{m_{n}}) × Aut(Sym_{6})^{m_{6}}.

Moreover, by definition, we have H_{3} ≃ \overline{∏}_{n≥3}Sym_{m_{n}}.

As a special case, if all but finitely many terms in m are 0, then (by putting |m| = ∑_{n} m_{n} < ∞) we have

\begin{aligned}|H_{1}| & = 2^{m_{2}(|m|−m_{1}−m_{2})} ∏_{i=0}^{m_{2}−1}(2^{m_{2}} − 2^{i}) = 2^{m_{2}(|m|−m_{1}−m_{2})+\binom{m_{2}}{2}} ∏_{i=1}^{m_{2}}(2^{i} − 1), \\ |H_{2}| & = 2^{m_{6}} ∏_{n≥3}(n!)^{m_{n}}, |H_{3}| = ∏_{n≥3} m_{n}!. \end{aligned}

Hence we have

|Aut(W_{m})| = |H_{1}| · |H_{2}| · |H_{3}|

= 2^{m_{2}(|m|−m_{1}−m_{2})+\binom{m_{2}}{2}+m_{6}} ∏_{i=1}^{m_{2}}(2^{i} − 1) ∏_{n≥3} ((n!)^{m_{n}}m_{n}!)

= (2^{m_{2}(|m|−m_{1}−m_{2}−1)+\binom{m_{2}}{2}+m_{6}} ∏_{i=1}^{m_{2}}(2^{i} − 1) ∏_{n≥3} m_{n}!)|W_{m}|.

4 Centralizers of normal subgroups generated by involutions

4.1 Proof of Theorem 3.1

In this section, we prove Theorem 3.1. From now, (W, S) always denotes a Coxeter system. In the proof, we use the notion of core subgroups (cf. Section 2.1). For a subgroup G ≤ W, let X_{G} be the set of all elements in G of the form w_{0}(I) (I ⊂ S) such that 1 ≠ w_{0}(I) ∈ Z(W_{I}). Then we have the following relation (proved below):

Proposition 4.1. Let H ◁ W be a normal subgroup generated by involutions. Then H is the smallest normal subgroup of W containing X_{H}, and

Z_{W} (H) = ⋂_{w_{0}(I)∈X_{H}} Core_{W} (N_{W} (W_{I})).

On the other hand, the subgroups Core_{W} (N_{W} (W_{I})) are determined completely (for irreducible (W, S)) by the following theorem, which we prove in later subsections. Here we use the notation (W (D_{3}), S(D_{3})) instead of (W (A_{3}), S(A_{3})).

Theorem 4.2. (See Definitions 2.5 and 2.8 for notations.) Let (W, S) be an irreducible Coxeter system of an arbitrary rank, and I nonempty proper subset of S. Then:

(i) If (W, S) = (W (B_{n}), S(B_{n})), 1 ≤ k < n ≤ ∞, τ ∈ Aut(Γ (B_{n})) and I = τ(S(B_{k})), then Core_{W} (N_{W} (W_{I})) = τ(G_{B_{n}}). (ii) If (W, S) = (W (D_{n}), S(D_{n})), 2 ≤ k < n ≤ ∞, τ ∈ Aut(Γ (D_{n})) and I = τ(S(D_{k})), then Core_{W} (N_{W} (W_{I})) = τ(G_{D_{n}}). (iii) Otherwise, Core_{W} (N_{W} (W_{I})) = Z(W). (cf. Lemma 2.12 for definition of G_{B_{n}} and G_{D_{n}}.)

Note that, Core_{W} (N_{W} (W_{I})) = N_{W} (W_{I}) = W if I = ∅ or S. Theorem 3.1 will be proved by combining Proposition 4.1 and Theorem 4.2.

In the proof of Proposition 4.1, we use the following two results:

Theorem 4.3 ([10], Theorem A). Let w be an involution in W. Then w is conjugate in W to some element w_{0}(I) (I ⊂ S) such that w_{0}(I) ∈ Z(W_{I}).

Lemma 4.4. Let W_{I} be a finite parabolic subgroup of W such that w_{0}(I) ∈ Z(W_{I}). Then Z_{W} (w_{0}(I)) = N_{W} (W_{I}).

Proof. First, assume u ∈ Z_{W} (w_{0}(I)). Then u^{−1}w_{0}(I)u = w_{0}(I) ∈ Z(W_{I}) and so w_{0}(I) · (u · α_{s}) = uw_{0}(I) · α_{s} = −u · α_{s} for all s ∈ I. This implies that u · α_{s} ∈ Φ_{I} for all s ∈ I, so that u ∈ N_{W} (W_{I}) by (2.16).

Conversely, assume u ∈ N_{W} (W_{I}). Put u′ = uw_{0}(I)u^{−1} ∈ W_{I}. Then we have u′ · α_{s} = −α_{s} for all s ∈ I (since w_{0}(I) maps u^{−1} · α_{s} ∈ Φ_{I} (cf. (2.16)) to −u^{−1} · α_{s}). Hence we have u′ = w_{0}(I) and so u ∈ Z_{W} (w_{0}(I)).

Proof of Proposition 4.1. By Theorem 4.3, every involution in H is conjugate to some element of X_{H} (since H ◁ W). This implies that any normal subgroup of W containing X_{H} also contains all the generators of H. Thus the first claim follows. For the second one, apply Lemmas 2.7 and 4.4.

Proof of Theorem 3.1. The claim (i) is obvious. From now, we assume H ⊄ Z(W). Note that Z(W) ⊂ Z_{W} (H). Note also that, by Proposition 4.1,

Z_{W} (H) ⊂ Core_{W} (N_{W} (W_{I})) for all w_{0}(I) ∈ X_{H}.  (4.1)

Case 1. (W, S) = (W (B_{n}), S(B_{n})), n ≥ 2 or (W (D_{n}), S(D_{n})), n ≥ 3: Let T = B, L = 1 for the former case, T = D, L = 2 for the latter case.

Subcase 1-1. T = B, n ≠ 2 or T = D, n ≠ 4: Note that in this case, any automorphism of Γ (T_{n}) preserves the sets S(T_{k}), elements w_{0}(S(T_{k})) (k ≥ L) and so the subgroup G_{T_{n}}.

Subsubcase 1-1-1. H ⊂ G_{T_{n}}: This is a case (ii) or (iii) (for τ identity), and so we have to show Z_{W} (H) = G_{T_{n}}. The inclusion ⊃ holds since G_{T_{n}} is abelian. Conversely, since H ⊄ Z(W), X_{H} contains an element other than w_{0}(S), so that we have Z_{W} (H) ⊂ G_{T_{n}} by (4.1) and Theorem 4.2.

Subsubcase 1-1-2. H ⊄ G_{T_{n}}: By the above remark, this is actually not a case (ii) or (iii), so that we have to show Z_{W} (H) ⊂ Z(W). Now X_{H} contains an element w_{0}(I) such that I ≠ S(T_{k}) for any L ≤ k ≤ n, since otherwise H ⊂ G_{T_{n}} by Lemma 2.12. For this I, we have Core_{W} (N_{W} (W_{I})) = Z(W) by Theorem 4.2, so that the claim follows from (4.1).

Subcase 1-2. T = B, n = 2: Note that X_{H} ⊂ {s_{1}, s_{2}, w_{0}(S)} in this case. Moreover, X_{H} ⊄ {w_{0}(S)} since H ⊄ Z(W).

Subsubcase 1-2-1. s_{1} ∈ X_{H} and s_{2} ∉ X_{H}: In this case, we have X_{H} ⊂ {s_{1}, w_{0}(S)} and so H ⊂ G_{B_{2}} by Lemma 2.12. This is a case (ii) (for τ identity). Now we have G_{B_{2}} ⊂ Z_{W} (H) since G_{B_{2}} is abelian, while Z_{W} (H) ⊂ G_{B_{2}} by (4.1) and Theorem 4.2 (applying to {s_{1}} ⊂ S). Thus the claim holds.

Subsubcase 1-2-2. s_{1} ∉ X_{H} and s_{2} ∈ X_{H}: By symmetry, this is also a case (ii) (for the unique τ ≠ id_{S}) and the claim holds similarly.

Subsubcase 1-2-3. s_{1} ∈ X_{H} and s_{2} ∈ X_{H}: Note that H = W. This is not a case (ii) or (iii), and actually Z_{W} (H) = Z(W).

Subcase 1-3. T = D, n = 4: Note that (by definition)

X_{H} ⊂ {s_{1}, s_{2}, s_{3}, s_{4}, s_{1}s_{2}s_{4}, s_{1}s_{2}, s_{2}s_{4}, s_{4}s_{1}, w_{0}(S)}.

Subsubcase 1-3-1. X_{H} contains one of the first five elements: Now we have H ⊄ τ(G_{D_{4}}) for any τ, so that this is not a case (iii) and we have to show Z_{W} (H) ⊂ Z(W). This claim follows from (4.1) (applying to the element of X_{H} given in the hypothesis here) and Theorem 4.2.

Subsubcase 1-3-2. X_{H} contains at least two of the elements s_{1}s_{2}, s_{2}s_{4}, s_{4}s_{1}: Now we have H ⊄ τ(G_{D_{4}}) for any τ, so that this is not a case (iii) and we have to show Z_{W} (H) ⊂ Z(W). Let X_{H} contain two such elements s_{i}s_{j}, s_{j}s_{k}, and put I = {s_{i}, s_{j}}, J = {s_{j}, s_{k}}. Then we have

Core_{W} (N_{W} (W_{I})) ∩ Core_{W} (N_{W} (W_{J})) ⊂ Core_{W} (N_{W} (W_{{s_{j}}}))

by (2.4), (2.15) and (2.2). Thus we have Z_{W} (H) ⊂ Core_{W} (N_{W} (W_{{s_{j}}})) = Z(W) by (4.1) and Theorem 4.2.

Subsubcase 1-3-3. X_{H} contains none of the first five elements and at most one of s_{1}s_{2}, s_{2}s_{4}, s_{4}s_{1}: Note that X_{H} ⊄ {w_{0}(S)} since H ⊄ Z(W). Thus we have s_{i}s_{j} ∈ X_{H} ⊂ {s_{i}s_{j}, w_{0}(S)} for one of (i, j) = (1, 2), (2, 4), (4, 1). Lemma 2.12 implies that this is a case (iii) (namely H ⊂ τ(G_{D_{4}})), by taking τ ∈ Aut(Γ) mapping s_{1}, s_{2} to s_{i}, s_{j} respectively. Now τ(G_{D_{4}}) ⊂ Z_{W} (H) since τ(G_{D_{4}}) is abelian. Conversely, we have Core_{W} (N_{W} (W_{{s_{i},s_{j}}})) = τ(G_{D_{4}}) by Theorem 4.2, so that Z_{W} (H) ⊂ τ(G_{D_{4}}) by (4.1). Thus the claim holds.

Case 2. (W, S) ≄ (W (B_{n}), S(B_{n})) (n ≥ 2), (W (D_{n}), S(D_{n})) (n ≥ 3): This is not a case (ii) or (iii), so that we have to show Z_{W} (H) ⊂ Z(W). Since H ⊄ Z(W), X_{H} contains an element other than w_{0}(S), so that we have Z_{W} (H) ⊂ Z(W) by (4.1) and Theorem 4.2. Hence the proof is concluded.

4.2 Some lemmas

In the rest of this paper, we prove Theorem 4.2. In this subsection, we prepare some lemmas used in our proof. From now, we abbreviate the notation Core_{W} (N_{W} (W_{I})) to C_{I}.

First, by combining Lemma 2.22, (2.4) and (2.2), we have:

If I ⊂ J ⊂ S and J ∖ I ⊂ I^{⊥}, then C_{J} ∩ C_{I} ⊂ C_{J∖I}.  (4.2)

Lemma 4.5 (Expanding Lemma). If I ⊂ S and s ∈ S ∖ (I ∪ I^{⊥}), then C_{I} ⊂ C_{I∪{s}}.

Proof. It is enough (by (2.3)) to show that C_{I} ⊂ N_{W} (W_{I∪{s}}). Let w ∈ C_{I}. By the hypothesis, we have c = ⟨α_{s}, α_{t}⟩ < 0 for some t ∈ I. Now since sws ∈ C_{I} ⊂ N_{W} (W_{I}), we have sws · α_{t} ∈ Φ_{I} (by (2.16)) and so ws · α_{t} ∈ Φ_{I∪{s}}. On the other hand, we have ws · α_{t} = w · α_{t} − 2cw · α_{s}. Thus w · α_{s} ∈ Φ_{I∪{s}} since w · α_{t} ∈ Φ_{I} (by (2.16)). Hence we have w ∈ N_{W} (W_{I∪{s}}) by (2.16).

For s ∈ S and I ⊂ S, let d_{Γ} (s, I) = min{d_{Γ} (s, t) | t ∈ I} denote the distance from s to the set I in the Coxeter graph Γ of (W, S).

Lemma 4.6 (Cutting Lemma). Let (W, S) be irreducible, I ⊂ S and s ∈ S ∖ I. Then for d_{Γ} (s, I) < k < ∞, we have C_{I} ⊂ C_{J}, where J = {t ∈ I | d_{Γ} (s, t) ≥ k}.

Proof. It is enough (by (2.3) and (2.16)) to show that w · Φ_{J} ⊂ Φ_{J} (or equivalently, w ·Π_{J} ⊂ Φ_{J}) for all w ∈ C_{I}. Assume contrary that t ∈ J and w ·α_{t} ∉ Φ_{J}. Note that w · α_{t} ∈ Φ_{I} (by (2.16)) and so s ∉ supp(w · α_{t}). Then by definition of J, we have

(d =) d_{Γ} (s, supp(w · α_{t})) < k ≤ d_{Γ} (s, t).

Take a shortest path s_{0} = s, s_{1}, …, s_{d−1}, s_{d} ∈ supp(w · α_{t}) in Γ from s to the set supp(w · α_{t}). Then by the above inequality, we have s_{i} ∈ {t}^{⊥} for all 0 ≤ i ≤ d − 1. Put u = ss_{1} ⋯ s_{d−1} ∈ W. Then we have uwu^{−1} · α_{t} = uw · α_{t} and so (by (2.13))

supp(uwu^{−1} · α_{t}) = supp(w · α_{t}) ∪ {s, s_{1}, …, s_{d−1}} ⊄ I

(note that s ∉ I). On the other hand, we have uwu^{−1} ∈ C_{I} and so uwu^{−1} · α_{t} ∈ Φ_{I} (by (2.16)). This is a contradiction. Hence the claim holds.

Lemma 4.7 (Shifting Lemma). Suppose that s, t ∈ S are in the same connected component of the odd Coxeter graph Γ^{odd} of (W, S). Then C_{{s}} = C_{{t}}.

Proof. By definition of Γ^{odd}, and by symmetry, it is enough to show that C_{{s}} ⊂ C_{{t}} for any s, t such that m(s, t) = 2k + 1 is odd. Now by putting u = (st)^{k} ∈ W, we have t = usu^{−1}. Thus for w ∈ C_{{s}}, we have

wtw^{−1} = wusu^{−1}w^{−1} = u(u^{−1}wu)s(u^{−1}wu)^{−1}u^{−1} = usu^{−1} = t

since u^{−1}wu ∈ C_{{s}}. Thus w ∈ N_{W} (W_{{t}}). Hence the claim follows from (2.3).

Moreover, we have:

Lemma 4.8. Let (W, S) be irreducible and I a nontrivial proper subset of S. Then Core_{W} (W_{I}) = 1.

Proof. Assume contrary that 1 ≠ w ∈ Core_{W} (W_{I}) (so that w · Φ_{I} = Φ_{I} by (2.16)). Fix s ∈ S ∖ I and take γ ∈ Φ_{I}^{+} such that w · γ ∈ Φ_{I}^{−}.

Case 1. (d =) d_{Γ} (s, supp(γ)) ≤ d_{Γ} (s, supp(w · γ)): Take a shortest path s_{0} = s, s_{1}, …, s_{d−1}, s_{d} ∈ supp(γ) in Γ from s to the set supp(γ). Then by the above inequality, we have s_{i} ∉ supp(w · γ) for all 0 ≤ i ≤ d − 1. Put u = ss_{1} ⋯ s_{d−1} ∈ W. Then we have u · γ ∈ Φ^{+} (by (2.12)), supp(u · γ) = supp(γ) ∪ {s, s_{1}, …, s_{d−1}} ⊄ I (by (2.13)) and so u · γ ∈ Φ^{+} ∖ Φ_{I}. On the other hand, we have uwu^{−1} · (u · γ) = u · (w · γ) ∈ Φ^{−} (by (2.12)). This is a contradiction, since uwu^{−1} ∈ Core_{W} (W_{I}) ⊂ W_{I}.

Case 2. d_{Γ} (s, supp(γ)) > d_{Γ} (s, supp(w ·γ)): Now by applying Case 1 to the elements w^{−1} ∈ Core_{W} (W_{I}) and −w · γ ∈ Φ_{I} [w^{−1} ], we have a contradiction again. Hence the claim holds in any case.

Owing to Lemma 4.8, we have the following results:

If (W, S) is irreducible, |W | = ∞ and s ∈ S, then C_{S∖{s}} = 1.  (4.3)

If (W, S) is irreducible, J ⊊ S and I is an irreducible component of J such that |W_{I}| = ∞, then C_{J} = 1.

(4.4)

Indeed, for (4.3), Corollary 2.20 (iii) implies N_{W} (W_{S∖{s}}) = W_{S∖{s}} and so Lemma 4.8 proves the claim. For (4.4), we have N_{W} (W_{J}) ⊂ W_{I∪I^{⊥}} by Proposition 2.18 (ii), while Core_{W} (W_{I∪I^{⊥}}) = 1 by Lemma 4.8 since I ∪ I^{⊥} ⊊ S, hence the claim follows from (2.2).

4.3 Proof for finite case

In this subsection, we prove Theorem 4.2 for the case |W | < ∞. From now, we abbreviate often the terms “Expanding Lemma”, “Cutting Lemma”, “Shifting Lemma” to ‘EL’, ‘CL’, ‘SL’, respectively.

Lemma 4.9. Let (W, S) be irreducible, |W | < ∞ and s ∈ S. Suppose that no condition below is satisfied: (I) W = W (B_{n}), n ≥ 2, s = s_{1}, (II) W = W (B_{2}), s = s_{2}, (III) W = W (I_{2}(m)), m even. Then C_{{s}} = Z(W).

Proof. Since Z(W) ⊂ C_{{s}} and ⋂_{t∈S} N_{W} (W_{{t}}) = Z(W), it is enough to show that C_{{s}} ⊂ C_{{t}} for all t ∈ S.

Case 1. The odd Coxeter graph Γ^{odd} of (W, S) is connected: Then the claim follows from the Shifting Lemma.

Case 2. W = W (B_{n}), n ≥ 3 and s ≠ s_{1}: We have C_{{s}} =^{SL} C_{{s_{i}}} for all

2 ≤ i ≤ n, while C_{{s_{2}}} ⊂^{EL} C_{{s_{1},s_{2}}} ⊂^{CL} C_{{s_{1}}} (since n ≥ 3). Thus the claim holds. Case 3. W = W (F_{4}): By symmetry, we may assume s = s_{1} or s_{2}. Now we

have C_{{s_{1}}} =^{SL} C_{{s_{2}}} ⊂^{EL} C_{{s_{2},s_{3}}} ⊂^{CL} C_{{s_{3}}} =^{SL} C_{{s_{4}}}. Hence the claim holds.

Corollary 4.10. Let (W, S) be irreducible, |W | < ∞, s ∈ S and suppose that there is a unique vertex t of Γ farthest from s. Suppose further that W and t do not satisfy any of the three conditions (I)–(III) in Lemma 4.9. Then C_{S∖{s}} = Z(W).

Proof. Now we have C_{S∖{s}} ⊂^{CL} C_{{t}} by the choice of t. Then apply Lemma 4.9.

Lemma 4.11. Suppose that one of the following conditions is satisfied: (I) W = W (B_{3}), s = s_{2}, (II) W = W (D_{4}), s = s_{3}, (III) W = W (H_{3}), s = s_{2}, (IV) W = W (I_{2}(m)) (m ≥ 6 even), s ∈ S. Then C_{I} = Z(W), where I = S ∖ {s}.

Proof. By the hypothesis and Corollary 2.20 (ii), we have N_{W} (W_{I}) = W_{I} × Z(W). Now a direct computation shows that sW_{I}s ∩ N_{W} (W_{I}) = 1, so that W_{I} ∩ C_{I} = 1 by (2.5). Since Z(W) ⊂ C_{I}, we have C_{I} = Z(W).

Lemma 4.12. (i) If W = W (B_{n}), 1 ≤ n < ∞, then Core_{W} (G_{B_{n}}) = G_{B_{n}}. (ii) If W = W (D_{n}), 3 ≤ n < ∞, then Core_{W} (G_{D_{n}} ⋊ ⟨s_{1}⟩) = G_{D_{n}}.

Proof. The claim (i) is obvious, since G_{B_{n}} ◁ W (cf. Lemma 2.12). For (ii), we have G_{D_{n}} ⊂ Core_{W} (G_{D_{n}} ⋊⟨s_{1}⟩) since G_{D_{n}} ◁W, while s_{1} ∉ Core_{W} (G_{D_{n}} ⋊⟨s_{1}⟩) since (s_{1}s_{3})s_{1}(s_{1}s_{3})^{−1} = s_{3} ∉ G_{D_{n}} ⋊ ⟨s_{1}⟩. Thus the claim holds.

Proof of Theorem 4.2 (for finite W). Note that Z(W) ⊂ C_{I} by definition.

Case 1. (W, S) = (W (T_{n}), S(T_{n})) for T = B, n ≥ 3 or T = D, 3 ≤ n ≠ 4: Put L = 1 in the former case, L = 2 in the latter case. Note that in this case, any automorphism of Γ (T_{n}) preserves the sets S(T_{k}), elements w_{0}(S(T_{k})) (k ≥ L) and so the subgroup G_{T_{n}}.

Subcase 1-1. I = S(T_{k}) for some L ≤ k < n: This is a case (i) or (ii) of Theorem 4.2 (for τ identity), so that we have to show C_{I} = G_{T_{n}}. Note that

C_{S(T_{i})} ⊂^{EL} C_{S(T_{j})} ⊂^{CL} C_{S(T_{i})} and so C_{S(T_{i})} = C_{S(T_{j})} for all L ≤ i < j < n.

Thus we may assume I = S(T_{L}), and we have C_{I} ⊂ ⋂_{i=L}^{n−1} N_{W} (W_{S(T_{i})}). By Corollary 2.21, (2.3) and Lemma 4.12, we have C_{I} ⊂ G_{T_{n}}. Conversely, since G_{T_{n}} is abelian and contains w_{0}(I), we have G_{T_{n}} ⊂ Z_{W} (w_{0}(I)) = N_{W} (W_{I}) by Lemma 4.4. Thus G_{T_{n}} ⊂ C_{I} since G_{T_{n}} ◁ W. Hence C_{I} = G_{T_{n}}.

Subcase 1-2. I ≠ S(T_{k}) for all L ≤ k < n: By the above remark, this is not a case (i) or (ii), and so we have to show C_{I} ⊂ Z(W). Note that I ≠ S. Let M be the first index ≥ 1 such that s_{M} ∉ I, so that S(T_{M−1}) ⊂ I (where

we put S(T_{0}) = ∅). If T = D and M = 2, then we have C_{I} ⊂^{EL} C_{S∖{s_{M} }} since I ≠ ∅. Otherwise, there is some M < i ≤ n such that s_{i} ∈ I (since otherwise

we have I = S(T_{M−1}); a contradiction), and so M < n and C_{I} ⊂^{EL} C_{S∖{s_{M} }}. In any case, we may assume that I = S ∖ {s_{M}}. Now there are the following three cases:

Subsubcase 1-2-1. M ≤ L + 1: Note that M < n, and so (T_{n}, M) ≠ (D_{3}, 3). If T_{n} = B_{3} and M = 2, then C_{I} = Z(W) by Lemma 4.11. Otherwise, we have a unique vertex of Γ farthest from s; that is s_{3−M} if T_{n} = D_{3} and M ≤ 2, and s_{n} otherwise (note that T_{n} ≠ D_{4}). Thus C_{I} = Z(W) by Corollary 4.10.

Subsubcase 1-2-2. L + 2 ≤ M ≤ n − 2: This hypothesis implies that

C_{I} ⊂^{CL} C_{I∖{s_{M−1},s_{M+1}}} ⊂^{EL} C_{S∖{s_{M−1}}},

so that the claim follows inductively from the case of smaller M.

Subsubcase 1-2-3. L + 2 ≤ M = n − 1: Note that n ≥ L + 3 and I =

S(T_{n−2}) ∪ {s_{n}}. Now we have C_{I} ⊂^{CL} C_{S(T_{n−3})} ⊂^{EL} C_{S(T_{n−2})} and so C_{I} ⊂ C_{{s_{n}}}

by (4.2). Thus C_{I} ⊂ C_{{s_{n}}} = Z(W) by Lemma 4.9.

Case 2. (W, S) = (W (B_{2}), S(B_{2})): Since I is proper and nonempty, we have I = {s_{i}} (i = 1 or 2). This is a case (i), by taking τ = id_{S} (if i = 1), τ ≠ id_{S} (if i = 2). Now we have to show C_{I} = τ(G_{B_{2}}). We have C_{I} ⊂ N_{W} (W_{τ({s_{1}})}) = τ(G_{B_{2}}) by Corollary 2.21 (i). Conversely, we have τ(G_{B_{2}}) ⊂ C_{I} by a similar argument to Subcase 1-1. Thus C_{I} = τ(G_{B_{2}}).

Case 3. (W, S) = (W (D_{4}), S(D_{4})): Note that I is proper and nonempty. Subcase 3-1. |I| = 1: This is not a case (i) or (ii), so that we have to show

C_{I} ⊂ Z(W). This follows from Lemma 4.9.

Subcase 3-2. |I| = 2 and s_{3} ∈ I: This is also not a case (i) or (ii), so that

we have to show C_{I} ⊂ Z(W). Let I = {s_{3}, s_{i}}. Then we have C_{I} ⊂^{CL} C_{{s_{i}}}, while C_{{s_{i}}} = Z(W) by the previous case. Thus C_{I} ⊂ Z(W).

Subcase 3-3. |I| = 2 and s_{3} ∉ I: Note that there is τ ∈ Aut(Γ) such that τ(S(D_{2})) = I. This is a case (ii), so that we have to show C_{I} = τ(G_{D_{4}}).

By symmetry, we may assume τ = id_{S}. First, we have C_{I} ⊂^{EL} C_{S(D_{3})} and so C_{I} ⊂ ⋂_{i=2}^{3} N_{W} (W_{S(D_{i})}) = G_{D_{4}} ⋊ ⟨s_{1}⟩ by Corollary 2.21 (ii). Thus we have C_{I} ⊂ G_{D_{4}} by (2.3) and Lemma 4.12. Conversely, we have G_{D_{4}} ⊂ C_{I} by a similar argument to Subcase 1-1. Hence we have C_{I} = G_{D_{4}}.

Subcase 3-4. |I| = 3 and s_{3} ∈ I: Note that there is τ ∈ Aut(Γ) such that τ(S(D_{3})) = I. This is a case (ii), so that we have to show C_{I} = τ(G_{D_{4}}). By

symmetry, we may assume τ = id_{S}. Now we have C_{I} ⊂^{CL} C_{S(D_{2})} ⊂^{EL} C_{I}, while C_{S(D_{2})} = G_{D_{4}} by the previous subcase. Thus C_{I} = G_{D_{4}}.

Subcase 3-5. I = S ∖ {s_{3}}: This is not a case (i) or (ii), so that we have to show C_{I} ⊂ Z(W). This follows from Lemma 4.11.

Case 4. (W, S) ≄ (W (B_{n}), S(B_{n})) (n ≥ 2), (W (D_{n}), S(D_{n})) (n ≥ 3): This is not a case (i) or (ii), so that we have to show C_{I} ⊂ Z(W). Note that |S| ≥ 2 since I is proper and nonempty.

Subcase 4-1. |S| = 2: Namely, (W, S) = (W (T ), S(T )), T = A_{2} or I_{2}(m) (5 ≤ m < ∞), and |I| = 1. Then we have C_{I} = Z(W) by Lemma 4.11 (for the latter case, with m even) or Lemma 4.9 (the other cases).

Subcase 4-2. |S| = 3: Namely, (W, S) = (W (H_{3}), S(H_{3})) (note that

W (A_{3}) ≃ W (D_{3})). Now we have C_{I} ⊂^{EL} C_{S∖{s_{i}}} for some i, while C_{S∖{s_{i}}} = Z(W) by Lemma 4.11 (if i = 2) or Corollary 4.10 (if i ≠ 2). Thus C_{I} ⊂ Z(W).

Subcase 4-3. |S| ≥ 4: Namely, (W, S) = (W (T ), S(T )) for T = A_{n}

(n ≥ 4), E_{n} (n = 6, 7, 8), F_{4} or H_{4}. Now we have C_{I} ⊂^{EL} C_{S∖{s_{i}}} for some i. Thus we may assume I = S ∖ {s_{i}}.

Subsubcase 4-3-1. There is a unique vertex of Γ farthest from s_{i}: Now we have C_{I} = Z(W) by Corollary 4.10.

Subsubcase 4-3-2. There are at least two vertices of Γ farthest from s_{i}: Namely, we have (T , i) = (A_{2k+1}, k + 1) (k ≥ 2), (E_{6}, 2), (E_{6}, 4) or (E_{8}, 5). Now there are exactly two vertices s, t of Γ farthest from s_{i}, and there is a vertex ≠ s, t adjacent to s and not adjacent to t. This implies that

C_{I} ⊂^{CL} C_{{s,t}} ⊂^{CL} C_{{t}}, while C_{{t}} = Z(W) by Lemma 4.9. Thus C_{I} ⊂ Z(W). Hence the proof is concluded.

4.4 Proof for infinite case

In this subsection, we prove Theorem 4.2 in the case |W | = ∞. The key facts are (4.3) and (4.4).

In the proof, we use a characterization (Proposition 4.14) of certain infinite Coxeter systems, which is based on the characterization of connected Coxeter graphs of finite type. Before stating this, we prepare the following graphtheoretic lemma.

Lemma 4.13. Let G be a connected acyclic graph (i.e. a tree) on nonempty vertex set V (G) of an arbitrary cardinality (with no edge labels here).

(i) If all vertices of G have degree ≤ 2 and G has a terminal vertex (i.e. vertex of degree 1) s_{0}, then G ≃ Γ (A_{n}) (as unlabelled graphs) for some 1 ≤ n ≤ ∞. (ii) If s_{0} ∈ V (G) and all vertices of G except s_{0} have degree ≤ 2, then each connected component G′ of G ∖ {s_{0}} contains exactly one vertex s adjacent to s_{0}, G′ ≃ Γ (A_{n}) (as unlabelled graphs) for some 1 ≤ n ≤ ∞ and s is a terminal vertex of G′. (iii) If all vertices of G have degree 2, then G ≃ Γ (A_{∞,∞}) (as unlabelled graphs).

Proof. (i) By the hypothesis, for any s ∈ V (G), G contains a unique simple path P_{s} = (t_{s}^{(0)} = s_{0}, t_{s}^{(1)}, …, t_{s}^{(ℓ−1)}, t_{s}^{(ℓ)} = s) from s_{0} to s. Let ℓ(s) = ℓ, the length of P_{s}. Then for all s_{1}, s_{2} ∈ V (G), we have either P_{s_{1}} ⊂ P_{s_{2}} or P_{s_{2}} ⊂ P_{s_{1}}: Otherwise, for the first index k such that t_{s_{1}}^{(k)} ≠ t_{s_{2}}^{(k)}, the vertex t_{s_{1}}^{(k−1)} = t_{s_{2}}^{(k−1)} is adjacent to distinct vertices t_{s_{1}}^{(k)}, t_{s_{2}}^{(k)} (and t_{s_{1}}^{(k−2)} if k ≥ 2) but this is impossible by the hypothesis on the degree of t_{s_{1}}^{(k−1)}.

This observation shows that the map ℓ : V (G) → {0, 1, 2, … } is injective and satisfies that i ∈ ℓ(V (G)) whenever 0 ≤ i < j and j ∈ ℓ(V (G)). Thus the set V (G) is finite or countable. Moreover, it also implies that two vertices s_{1}, s_{2} are adjacent if ℓ(s_{1}) = ℓ(s_{2}) ± 1, while by definition of ℓ, these are not adjacent if ℓ(s_{1}) ≠ ℓ(s_{2}) ± 1. Thus the claim holds. (ii) First, take a vertex t of G′ and a simple path P in G from s_{0} to t. Then the vertex s of P next to s_{0} is adjacent to s_{0} and contained in G′. On the other hand, if G′ contains two vertices adjacent to s_{0}, then s_{0} and a path in G′ between these two vertices form a closed path in G. This is a contradiction, so that the first claim follows. Since s has degree ≤ 2 in G and adjacent to s_{0} ∉ V (G′), s is a terminal vertex of G′. Now the second claim is deduced by applying (i) to G′ and s. (iii) This follows from (ii), since G is nonempty and has no terminal vertices.

Proposition 4.14. Let (W, S) be an irreducible Coxeter system of an arbitrary rank, with Coxeter graph Γ. Suppose that |W | = ∞ and |W_{I}| < ∞ for all finite subsets I ⊂ S. Then Γ ≃ Γ (A_{∞}), Γ (B_{∞}), Γ (D_{∞}) or Γ (A_{∞,∞}).

Proof. In this proof, a full subgraph Γ_{I} of Γ is said to be forbidden if |I| < ∞ and |W_{I}| = ∞. The hypothesis means that |W | = ∞ and Γ is connected and contains no forbidden subgraphs. This implies |S| = ∞ immediately.

Step 1. Γ is acyclic: This follows immediately from the fact that any nontrivial cycle in Γ forms a forbidden subgraph.

Step 2. No s ∈ S has degree ≥ 4 in Γ: Otherwise, this s and the four adjacent vertices form a forbidden subgraph of Γ. This is a contradiction.

Step 3. At most one s ∈ S has degree 3 in Γ: Assume contrary that two distinct vertices s, t ∈ S have degree 3. Since Γ is connected, there is a path P in Γ between s and t. Then s, t, P and all the vertices adjacent to s or t form a forbidden subgraph. This is a contradiction.

Step 4. If some s ∈ S has degree 3 in Γ, then Γ ≃ Γ (D_{∞}): By Steps 1–3, we can apply Lemma 4.13 (ii) to this case. This lemma shows that Γ_{S∖{s}} consists of three connected components ≃ Γ (A_{n_{1}}), Γ (A_{n_{2}}), Γ (A_{n_{3}}) (as unlabelled graphs) respectively, of which a terminal vertex is adjacent to s in Γ. By symmetry, we may assume n_{1} ≥ n_{2} ≥ n_{3} ≥ 1.

Now we have n_{1} = ∞ since |S| = ∞. If n_{2} ≥ 2, then Γ must contain a forbidden subgraph (≃ Γ (Ẽ_{8}) as unlabelled graphs), but this is a contradiction. Thus we have n_{2} = n_{3} = 1 and so Γ ≃ Γ (D_{∞}) as unlabelled graphs. Moreover, every edge of Γ must have no label (or label ‘3’), since otherwise Γ must contain a forbidden subgraph again. Hence Γ ≃ Γ (D_{∞}) (as Coxeter graphs) in this case.

Step 5. If all vertices of Γ have degree ≤ 2, then Γ ≃ Γ (A_{∞}), Γ (B_{∞}) or Γ (A_{∞,∞}): First, we consider the case that Γ has a terminal vertex. Then Lemma 4.13 (i) implies that Γ ≃ Γ (A_{∞}) as unlabelled graphs (note that |S| = ∞). Moreover, by a similar argument to Step 4, the hypothesis (Γ contains no forbidden subgraphs) detects the edge-labels of Γ, so that we have Γ ≃ Γ (A_{∞}) or Γ (B_{∞}) (as Coxeter graphs). The other case is similar; we have Γ ≃ Γ (A_{∞,∞}) as Coxeter graphs by Lemma 4.13 (iii) and the hypothesis. Hence the proof is concluded.

Proof of Theorem 4.2 (for infinite W). Note that Z(W) = 1 in this case.

Case 1. (W, S) = (W (T_{n}), S(T_{n})) for T_{n} = A_{∞}, B_{∞}, D_{∞} or A_{∞,∞}: Put L = 1 if T_{n} = B_{∞}, L = 2 if T_{n} = D_{∞}. Moreover, for k ≥ 1, put

J_{k} = {s_{1}, s_{2}, …, s_{k}} if T_{n} ≠ A_{∞,∞}, J_{k} = {s_{−k}, s_{−k+1}, …, s_{k}} if T_{n} = A_{∞,∞}.

Subcase 1-1. T_{n} = B_{∞} or D_{∞}, and I = S(T_{k}) for some L ≤ k < ∞: This is a case (i) or (ii) (for τ identity), so that we have to show C_{I} = G_{T_{∞}}. Put G_{i} = W_{J_{k+i}} and H_{i} = N_{G_{i}}(W_{I}) for i ≥ 1. Then we have ⋃_{i=1}^{∞} G_{i} = W and ⋃_{i=1}^{∞} H_{i} = N_{W} (W_{I}), so that C_{I} ⊂ ⋃_{i=1}^{∞} Core_{G_{i}}(H_{i}) by Lemma 2.6. Moreover, by the result of finite case (Section 4.3), we have Core_{G_{i}}(H_{i}) = G_{T_{k+i}} for all i ≥ 1. Since ⋃_{i=1}^{∞} G_{T_{k+i}} = G_{T_{∞}} (cf. Lemma 2.12), we have C_{I} ⊂ G_{T_{∞}}.

On the other hand, we have C_{S(T_{L})} ⊂^{EL} C_{I}, while G_{T_{∞}} ⊂ Z_{W} (w_{0}(S(T_{L}))) since w_{0}(S(T_{L})) ∈ G_{T_{∞}} and G_{T_{∞}} is abelian. Thus G_{T_{∞}} ⊂ N_{W} (W_{S(T_{L})}) by Lemma 4.4, G_{T_{∞}} ⊂ C_{S(T_{L})} by (2.3) and so G_{T_{∞}} ⊂ C_{I}. Hence C_{I} = G_{T_{∞}}.

Subcase 1-2. The hypothesis of Subcase 1-1 is not satisfied: This is not a case (i) or (ii), so that we have to show C_{I} = 1.

Subsubcase 1-2-1. |I| < ∞: Let w ∈ C_{I}. Now take a sufficiently large 4 ≤ k < ∞ so that I ⊂ J_{k} and w ∈ W_{J_{k}}. Put G_{i} = W_{J_{k+i}} and H_{i} = N_{G_{i}}(W_{I}) for i ≥ 1, so that ⋃_{i=1}^{∞} G_{i} = W and ⋃_{i=1}^{∞} H_{i} = N_{W} (W_{I}). Now by the hypothesis of Subcase 1-2, and by the result for finite case (Section 4.3), we have Core_{G_{i}}(H_{i}) ⊂ Z(G_{i}) ⊂ {1, w_{0}(J_{k+i})} for all i. Moreover, by Lemma 2.6, we have C_{I} ⊂ ⋃_{i=1}^{∞} Core_{G_{i}}(H_{i}). Since w_{0}(J_{k+i}) ∉ W_{J_{k}} for any i ≥ 1, this implies that w = 1 by the choice of k. Hence we have C_{I} = 1.

Subsubcase 1-2-2. |I| = ∞: If I has an irreducible component J of infinite cardinality, then C_{I} = 1 by (4.4). Thus we may assume that I is a union of infinitely many irreducible components of finite cardinality. Now we can choose indices 4 ≤ i ≤ j < ∞ so that s_{k} ∉ I for all i ≤ k ≤ j, s_{i−1} ∈ I and s_{j+1} ∈ I. Let K_{1}, K_{2} be the (distinct) irreducible components of I containing s_{i−1}, s_{j+1}

respectively. Then we have C_{I} ⊂^{CL} C_{I∖(K_{1}∪K_{2})} and so C_{I} ⊂ C_{K_{1}∪K_{2}} by (4.2). Moreover, we have C_{K_{1}∪K_{2}} = 1 by Subsubcase 1-2-1. Thus C_{I} = 1.

Case 2. (W, S) ≄ (W (T ), S(T )) for T = A_{∞}, B_{∞}, D_{∞}, A_{∞,∞}: This is not a case (i) or (ii), so that we have to show C_{I} = 1. By Proposition 4.14, there is a finite subset J_{0} ⊂ S such that |W_{J_{0}}| = ∞. This J_{0} consists of only finitely many irreducible components, and so we have |W_{J}| = ∞ for some irreducible component of J_{0}. Since Γ is connected and |J| < ∞, there is a (finite) sequence s_{1}, s_{2}, …, s_{r} of elements of S such that s_{i} ∉ I_{i−1} ∪ I_{i−1}^{⊥} for all 1 ≤ i ≤ r and J ⊂ I_{r}, where we put I_{0} = I and I_{i} = I_{i−1} ∪ {s_{i}} (1 ≤ i ≤ r) inductively. Now

we have C_{I_{i−1}} ⊂^{EL} C_{I_{i}} for all 1 ≤ i ≤ r, so that C_{I} ⊂ C_{I_{r−1}} and C_{I} ⊂ C_{I_{r}}.

Subcase 2-1. I_{r} ≠ S: Now an irreducible component of I_{r} (namely, the one containing J) generates an infinite group. Thus C_{I} ⊂ C_{I_{r}} = 1 by (4.4).

Subcase 2-2. I_{r} = S: Note that r ≥ 1 since I is proper. Since (W, S) is irreducible, we have C_{I} ⊂ C_{I_{r−1}} = 1 by (4.3). Hence the proof is concluded.

Bibliography

[1] Brink, B., Howlett, R. B. Normalizers of parabolic subgroups in Coxeter groups. Invent. Math. 136: 323-351.

[2] Deodhar, V. V. On the root system of a Coxeter group. Comm. Algebra 10: 611-630.

[3] Franzsen, W. N., Howlett, R. B. Automorphisms of nearly finite Coxeter groups. Adv. Geom. 3: 301-338.

[4] Howlett, R. B. Normalizers of parabolic subgroups of reflection groups. J. London Math. Soc. (2) 21: 62-80.

[5] Humphreys, J. E. (1990). Reflection Groups and Coxeter Groups. Cambridge Univ. Press.

[6] Krammer, D. (1994). The conjugacy problem for Coxeter groups. Ph. D. Thesis, Universiteit Utrecht.

[7] Mihalik, M., Ratcliffe, J., Tschantz, S. On the Isomorphism Problem for finitely generated Coxeter groups. I, Basic matching. arXiv:math.GR/0501075.

[8] Nuida, K. On centralizers of parabolic subgroups in Coxeter groups. arXiv:math.GR/0501061.

[9] Paris, L. Irreducible Coxeter groups. arXiv:math.GR/0412214.

[10] Richardson, R. W. Conjugacy classes of involutions in Coxeter groups. Bull. Austral. Math. Soc. 26: 1-15.

Part II

Almost central involutions

in split extensions of Coxeter groups by graph

automorphisms

ALMOST CENTRAL INVOLUTIONS IN SPLIT EXTENSIONS OF COXETER GROUPS BY GRAPH AUTOMORPHISMS

KOJI NUIDA

In this paper, given a split extension of an arbitrary Coxeter group by automorphisms of the Coxeter graph, we determine the involutions in that extension whose centralizer has finite index. Our result has applications to many problems such as the isomorphism problem of general Coxeter groups. In the argument, some properties of certain special elements and of the fixed-point subgroups by graph automorphisms in Coxeter groups, which are of independent interest, are also given.

1 Introduction

Let (W, S) be an arbitrary Coxeter system, possibly of infinite rank, and G a group acting on W. We assume that the action of G preserves the set S; namely, each element of G gives rise to an automorphism of the Coxeter graph of (W, S). The subject of this paper is the almost central involutions in the semidirect product W ⋊ G corresponding the action of G; that is, involutions which is central in some subgroup of W ⋊ G of finite index. We determine those involutions in W ⋊ G, hence the subgroup generated by those involutions, in terms of the structure of the Coxeter system (W, S) and the action of G on W (Theorem 3.1). Actually, this subgroup is the product of some finite irreducible components of W, specified in terms of the action of G, and a subgroup of G. Note that this subgroup is determined by the group structure of W ⋊ G only, so our result can extract some information on the Coxeter group W from the group structure of W ⋊ G. Moreover, if W ⋊ G admits another expression W′ ⋊ G′ of this type, our result exhibits some relation between the Coxeter groups W and W′ through the subgroup in problem (Theorem 3.2).

The main motive of this research is an application to the isomorphism problem of general Coxeter groups; that is, the problem of deciding which Coxeter groups are isomorphic as abstract groups. An important phase of the problem is to determine whether a given group isomorphism f between two Coxeter groups W and W′ maps the reflections in W onto those of W′. As summarized in Section 3.3, it is shown by a result of the author’s preceding paper [14] that both the centralizer of a reflection t in W and that of f(t) in W′ are semidirect products satisfying the hypothesis of our main theorem. Since those centralizers are isomorphic via f, our main theorem can derive some properties of f(t) from those of W and of t. In particular, f(t) is a reflection in W′ whenever W and t satisfy a certain condition which is independent on the choice of W′ and f (Theorem 3.7). When the condition is actually satisfied will be investigated in a forthcoming paper [13] of the author. Note that this argument works without any assumption on finiteness of ranks of W or of W′, in contrast with most of the preceding results on the isomorphism problem which covers the case of finite ranks only.

For other applications, our result implies that the product of all finite irreducible components of a Coxeter group W is independent on the choice of the generating set S of W (Example 3.3). On the other hand, regarding certain semidirect product decompositions of W into two Coxeter groups which arise from the partition of S into conjugacy classes, our result shows that, under a certain condition, the normal factor possesses no finite irreducible component (Example 3.6). See Section 3.2 for further examples.

This paper is organized as follows. Section 2 is a preliminary for basics and further remarks on abstract groups and Coxeter groups. Section 3 summarizes the main result and its applications mentioned above. In Section 4, we recall the notion of essential elements in Coxeter groups introduced by Daan Krammer [10], and summarize some properties studied by Krammer and by Luis Paris [16]. In Section 5, we give some results on the fixed-point subgroup of a Coxeter group by an automorphism of the Coxeter graph, together with preceding results given by Robert Steinberg [18], by Bernhard Mühlherr [11] and by Masayuki Nanba [12]. Finally, Section 6 is devoted to the proof of the main theorem. Acknowledgement. The author would like to express his deep gratitude to everyone who helped him, especially to his supervisor Itaru Terada and also to Kazuhiko Koike for their precious advice and encouragement. The author had been supported by JSPS Research Fellowship throughout this research.

2 Preliminaries

2.1 On abstract groups

In this subsection, we fix notations for abstract groups, and give some definitions and facts. Let G be an arbitrary group. We denote H ≤ G if H is a subgroup of G, and H ⊴ G if H is a normal subgroup of G. For a subset X ⊆ G, let ⟨X⟩ and ⟨X⟩_{◁G} denote the subgroup and the normal subgroup, respectively, of G generated by X. Put

Z_{H}(X) = {g ∈ H | gx = xg for all x ∈ X} for H ≤ G,

so Z_{G}(X) is the centralizer of X in G. Write

x^{g} = g^{−1}xg and X^{g} = {x^{g} | x ∈ X} for g, x ∈ G and X ⊆ G.

For H ≤ G, put

Core_{G}H = ⋂_{g∈G} H^{g},

the core of H in G. It is easily verified that Core_{G}H is the unique largest normal subgroup of G contained in H.

Lemma 2.1. Let G be a group.

1. If H ⊴ G, then Z_{G}(H) ⊴ G.

2. If X ⊆ G, then Z_{G}(⟨X⟩_{◁G}) = Core_{G}Z_{G}(X).

Proof. The proof of (1) is straightforward. For (2), the inclusion ⊆ follows from (1) since ⟨X⟩ ⊆ ⟨X⟩_{◁G}, so it suffices to show that H ⊆ Z_{G}(⟨X⟩_{◁G}) whenever H ⊴ G and H ⊆ Z_{G}(X). Now we have X ⊆ Z_{G}(H) ⊴ G by (1), so ⟨X⟩_{◁G} ⊆ Z_{G}(H), proving the claim.

Let [G : H] denote the index of a subgroup H ≤ G in G. Recall the following well-known properties:

if G ≥ H_{1} ≥ H_{2}, then [G : H_{2}] = [G : H_{1}] [H_{1} : H_{2}] ;  (2.1)
if H_{1}, H_{2} ≤ G, then [G : H_{1}] ≥ [H_{2} : H_{1} ∩ H_{2}] .  (2.2)

From these properties it is easy to deduce that

if H_{1}, H_{2} ≤ G and [G : H_{2}] < ∞, then the followings are equivalent:

[G : H_{1}] < ∞; [G : H_{1} ∩ H_{2}] < ∞; [H_{2} : H_{1} ∩ H_{2}] < ∞.  (2.3)

Lemma 2.2. Let H ≤ G. Then [G : H] < ∞ if and only if [G : Core_{G}H] < ∞.

Proof. The only nontrivial part is the “only if” part. Let G = ⨆_{i=1}^{n} Hg_{i} (where n = [G : H] < ∞) be a decomposition into cosets. Then Core_{G}H = ⋂_{i=1}^{n} H^{g_{i}}. Now for 1 ≤ k ≤ n, two subgroups H^{g_{k}} and H have the same (finite) index in G, so the subgroup ⋂_{i=1}^{k} H^{g_{i}} has finite index in ⋂_{i=1}^{k−1} H^{g_{i}} by (2.2). Now iterative use of (2.1) yields the desired conclusion.

We say that an element g ∈ G is almost central in G if [G : Z_{G}(g)] < ∞.

Corollary 2.3. Let G be a group and g ∈ G.

1. We have [G : Z_{G}(⟨g⟩_{◁G})] < ∞ if and only if g is almost central in G.

2. If g is almost central in G, then all h ∈ ⟨g⟩_{◁G} are almost central in G.

Proof. The claim (1) follows immediately from Lemmas 2.1 (2) and 2.2, and (2) is a consequence of (1) and the observation Z_{G}(h) ≥ Z_{G}(⟨g⟩_{◁G}).

Lemma 2.4. Let G_{1} ⋊ G_{2} be a semidirect product of two groups, and suppose that H_{i} ≤ G_{i} has finite index in G_{i} for i = 1, 2. Then [G_{1} ⋊ G_{2} : H_{1}H_{2}] < ∞.

Proof. Decompose G_{i} as ⨆_{j=1}^{r_{i}} g_{i,j}H_{i}, where r_{i} < ∞. Then

G_{1} ⋊ G_{2} = ⋃_{1≤j≤r_{1}, 1≤k≤r_{2}} g_{1,j}H_{1}g_{2,k}H_{2} = ⋃_{j,k} g_{1,j}g_{2,k}H_{1}^{g_{2,k}}H_{2}.

Since [G_{1} : H_{1}] < ∞, we have [H_{1}^{g_{2,k}} : H_{1}^{g_{2,k}} ∩ H_{1}] < ∞ by (2.2). Let H_{1}^{g_{2,k}} = ⨆_{ℓ=1}^{n_{k}} h_{k,ℓ}(H_{1}^{g_{2,k}} ∩ H_{1}) (where n_{k} < ∞) be the corresponding coset decomposition. Then we have

G_{1} ⋊ G_{2} = ⋃_{j,k} ⋃_{ℓ=1}^{n_{k}} g_{1,j}g_{2,k}h_{k,ℓ}(H_{1}^{g_{2,k}} ∩ H_{1})H_{2} ⊆ ⋃_{j,k,ℓ} g_{1,j}g_{2,k}h_{k,ℓ}H_{1}H_{2},

where the last union is taken over the finite set of the (j, k, ℓ), as desired.

2.2 Coxeter groups

This subsection summarizes some basic definitions and facts for Coxeter groups, which are found in the book [9] unless otherwise noticed, and give further results and remarks. Some more preliminaries focusing into the two topics, essential elements and fixed-point subgroups by Coxeter graph automorphisms, will be given in Sections 4 and 5.

Definitions

A pair (W, S) of a group W and its generating set S is called a Coxeter system if W admits the following presentation

W = ⟨S | (st)^{m_{s,t}} = 1 for all s, t ∈ S such that m_{s,t} < ∞⟩,

where the m_{s,t} ∈ {1, 2, … } ∪ {∞} are symmetric in s, t ∈ S, and m_{s,t} = 1 if and only if s = t. A group W is called a Coxeter group if some S ⊆ W makes (W, S) a Coxeter system. The cardinality |S| of S is called the rank of (W, S) or of W, which is not assumed to be finite unless otherwise noticed. Now m_{s,t} coincides with the order of st ∈ W, so the system (W, S) determines uniquely (up to isomorphism) the Coxeter graph denoted by Γ, that is a simple unoriented graph with vertex set S in which every two vertices s, t ∈ S is joined by an edge with label m_{s,t} if and only if m_{s,t} ≥ 3. (By convention, the label ‘3’ is usually omitted when drawing a picture.)

An automorphism of the Coxeter graph Γ is briefly called a graph automorphism of (W, S) or of W. Let Aut Γ denote the set of the graph automorphisms of W. Then m_{τ(s),τ(t)} = m_{s,t} for τ ∈ Aut Γ and s, t ∈ S, so this τ extends uniquely to an automorphism of the group W denoted also by τ.

For I ⊆ S, let W_{I} denote the standard parabolic subgroup ⟨I⟩ of W generated by I. A subgroup conjugate to some W_{I} is called a parabolic subgroup. (In some context, the term “parabolic subgroups” signifies the subgroups W_{I} themselves only.) Now (W_{I}, I) is also a Coxeter system, of which the Coxeter graph Γ_{I} is the full subgraph of Γ with vertex set I. If I is (the vertex set of) a connected component of Γ, then W_{I} is called an irreducible component of (W, S) (or of W, if the set S is obvious from the context). If Γ is connected, then (W, S) and W are called irreducible. Now W is the (restricted) direct product of the irreducible components; however, each irreducible component is not necessarily directly indecomposable as an abstract group.

Regarding the standard parabolic subgroups, it is well known that

if I, J ⊆ S, then W_{I} ∩ W_{J} = W_{I∩J}.  (2.4)

Then, since each w ∈ W is a product of a finite number of elements of S, it follows that W possesses a unique minimal standard parabolic subgroup containing w, called the standard parabolic closure of w and denoted here by SP(w). Now the support supp(w) ⊆ S of w ∈ W is defined by

W_{supp(w)} = SP(w).

On the other hand, we have the following fact for parabolic subgroups:

Proposition 2.5 (See e.g. [7, Corollary 7]). Let I, J ⊆ S and w ∈ W. Then W_{I} ∩ (W_{J})^{w} = (W_{K})^{u} for some K ⊆ I and u ∈ W_{I}. Moreover, we have K ≠ I whenever W_{I} ≠ (W_{J})^{w}.

This proposition denies the existence of an infinite, properly descending sequence (W_{I_{1}})^{w_{1}} ⊃ (W_{I_{2}})^{w_{2}} ⊃ ⋯ of parabolic subgroups with I_{1} finite, since it enables us to choose the I_{i} inductively as descending properly. Thus W also possesses a unique minimal parabolic subgroup containing a given w ∈ W, called the parabolic closure of w and denoted here by P(w).

Let ℓ denote the length function of (W, S), namely ℓ(w) (where w ∈ W) is the minimal length n of an expression w = s_{1} ⋯ s_{n} with s_{i} ∈ S (so ℓ(w^{−1}) = ℓ(w)). Such an expression of w with n = ℓ(w) is called a reduced expression. The following three well-known properties will be used in the arguments below, without references:

if w ∈ W and s ∈ S, then ℓ(ws) = ℓ(w) ± 1;
for I ⊆ S, the length function ℓ_{I} of (W_{I}, I) agrees with ℓ on W_{I};
supp(w) = {s_{1}, …, s_{n}} for any reduced expression w = s_{1} ⋯ s_{n}.

Theorem 2.6 (Exchange Condition). Let w = s_{1} ⋯ s_{n} ∈ W, s_{i} ∈ S and t ∈ S with ℓ(wt) < ℓ(w). Then there exists an index i such that wt = s_{1} ⋯ ŝ_{i} ⋯ s_{n} (s_{i} omitted).

Geometric representation and root systems

Let V denote the geometric representation space of W, that is an ℝ-vector space equipped with the basis Π = {α_{s} | s ∈ S} and the symmetric bilinear form ⟨ , ⟩ determined by

⟨α_{s}, α_{t}⟩ = − cos \frac{π}{m_{s,t}} if m_{s,t} < ∞ and ⟨α_{s}, α_{t}⟩ = −1 if m_{s,t} = ∞.

W acts faithfully on V by s · v = v − 2⟨α_{s}, v⟩α_{s} for s ∈ S and v ∈ V, making ⟨ , ⟩ W -invariant. Let Φ = W · Π, Φ^{+} = Φ ∩ ℝ_{≥0}Π and Φ^{−} = −Φ^{+} denote, respectively, the root system, the set of positive roots and the set of negative roots. We have Φ = Φ^{+} ⊔ Φ^{−}, and Φ consists of unit vectors with respect to ⟨ , ⟩. For any subset Ψ ⊆ Φ and w ∈ W, write

Ψ^{+} = Ψ ∩ Φ^{+}, Ψ^{−} = Ψ ∩ Φ^{−} and Ψ [w] = {γ ∈ Ψ^{+} | w · γ ∈ Φ^{−}}.

Then ℓ(w) coincides with the cardinality |Φ [w] | of Φ [w], so w = 1 if and only if Φ [w] = ∅. This implies a further property that

for w, u ∈ W, we have w = u if and only if Φ [w] = Φ [u] .  (2.5)

For any v = ∑_{s∈S} c_{s}α_{s} ∈ V, the support supp(v) ⊆ S of v is defined by

supp(v) = {s ∈ S | c_{s} ≠ 0}.

For I ⊆ S, put

Π_{I} = {α_{s} | s ∈ I} ⊂ Π, V_{I} = span_{ℝ}Π_{I} ⊂ V and Φ_{I} = Φ ∩ V_{I}.

Then it is well known (see e.g. [8, Lemma 4]) that

Φ_{I} = W_{I} · Π_{I},  (2.6)

the root system of a Coxeter system (W_{I}, I). Note that Φ [w] ⊆ Φ_{supp(w)} for w ∈ W. Moreover, it is well known that for I ⊆ S, any w ∈ W admits a unique decomposition w = w^{I}w_{I} with w_{I} ∈ W_{I} and Φ_{I} [w^{I}] = ∅. Note that Φ [w_{I}] = Φ_{I} [w]. This implies that

if w ∈ W and s ∈ supp(w), then s ∈ supp(γ) for some γ ∈ Φ [w]  (2.7)

(if this fails, then Φ [w] = Φ_{I} [w] = Φ [w_{I}] where I = supp(w) ∖ {s}, so w = w_{I} ∈ W_{I} by (2.5), contradicting the definition of supp(w)). Now we prepare a technical lemma which will be used in later sections.

Lemma 2.7. Let 1 ≠ w ∈ W and I = supp(w) ⊆ S.

1. Let γ ∈ Φ^{+}, J = supp(γ) and suppose that I ∩ J = ∅ and J is adjacent to I in the Coxeter graph Γ. Then w · γ ∈ Φ_{I∪J}^{+} ∖ Φ_{J}.

2. Suppose that s ∈ S ∖ I is adjacent to I in Γ. Then supp(w^{s}) = I ∪ {s}.

3. For i = 1, 2, let 1 ≠ u_{i} ∈ W, J_{i} = supp(u_{i}) and suppose that J_{i} ∩ I = ∅ and J_{2} is adjacent to I in Γ. Then u_{1}wu_{2} ≠ w.

Proof. (1) Since the action of w ∈ W_{I} leaves the coefficient in γ of any α_{s} ∈ Π_{J} unchanged, it suffices to show that w · γ ≠ γ. Take s ∈ I adjacent to J, and β ∈ Φ_{I}^{+} such that s ∈ supp(β) and w · β ∈ Φ_{I}^{−} (see (2.7)). This choice yields that ⟨β, γ⟩ < 0 and ⟨w · β, γ⟩ ≥ 0 since I ∩ J = ∅, showing that w · γ ≠ γ since ⟨ , ⟩ is W -invariant.

(2) Put J = supp(w^{s}). Then we have w = (w^{s})^{s} ∈ W_{J∪{s}} and so I ⊆ J ∪ {s}, therefore I ⊆ J since s ∉ I. On the other hand, ws·α_{s} = −w ·α_{s} ∈ Φ^{−} ∖{−α_{s}} by (1), so we have w^{s} · α_{s} ∈ Φ^{−} and s ∈ J. Thus we have I ∪ {s} ⊆ J, while w^{s} ∈ W_{I∪{s}}, proving the claim.

(3) Take s ∈ J_{2} adjacent to I, and γ ∈ Φ [u_{2}^{−1} ] ⊆ Φ_{J_{2}}^{+} with s ∈ supp(γ) (see (2.7)), so β = u_{2}^{−1} · γ lies in Φ_{J_{2}}^{−}. Then w · β ∈ Φ^{−} since I ∩ J_{2} = ∅, while wu_{2} · β = w · γ ∈ Φ^{+} ∖ Φ_{S∖I} by (1) and so u_{1}wu_{2} · β ∈ Φ^{+} since J_{1} ∩ I = ∅. Thus we have u_{1}wu_{2} ≠ w as desired.

For γ = w · α_{s} ∈ Φ, let s_{γ} = wsw^{−1} denote the reflection along the root γ acting on V by s_{γ} · v = v − 2⟨γ, v⟩γ for v ∈ V. Let

S^{W} = ⋃_{w∈W} wSw^{−1}

denote the set of the reflections in W, which depends on the set S in general.

Lemma 2.8. Let W be an infinite irreducible Coxeter group. Then the orbit W · γ ⊆ Φ of any root γ ∈ Φ is an infinite set.

The proof of this lemma requires the following two results:

Proposition 2.9 ([4, proof of Proposition 4.2]). Let W be an infinite irreducible Coxeter group of finite rank, and I ⊂ S a proper subset. Then |Φ ∖ Φ_{I}| = ∞.

Proposition 2.10 ([15, Lemma 2.9]). Let w ∈ W and suppose that I, J ⊆ S are disjoint subsets such that w · Π_{I} = Π_{I} and w · Π_{J} ⊆ Φ^{−}. Then we have Φ_{I∪J} [w] = Φ_{I∪J}^{+} ∖ Φ_{I}.

Proof of Lemma 2.8. First we show that, for any β ∈ Φ^{+}, we have ⟨β, α_{s}⟩ < 0 for some s ∈ S. This is obvious if |S| = ∞ (choose s ∈ S ∖ supp(β) adjacent in the infinite connected graph Γ to the finite set supp(β)), so suppose that |S| < ∞. Assume contrary that ⟨β, α_{s}⟩ ≥ 0 for all s ∈ S. Put I = {s ∈ S | ⟨β, α_{s}⟩ = 0} ≠ S (note that ⟨β, β⟩ = 1), so s_{β} fixes Φ_{I} pointwise. Then for any s ∈ S ∖ I, we have ⟨β, α_{s}⟩ > 0 and s_{β} · α_{s} = α_{s} − 2⟨β, α_{s}⟩β ∈ Φ^{−}. Thus Proposition 2.10 implies that Φ [s_{β}] = Φ^{+} ∖ Φ_{I}, which has cardinality ℓ(s_{β}) < ∞, contradicting Proposition 2.9. Hence the claim of this paragraph holds.

For the lemma, we may assume that γ ∈ Φ^{+}. Then by taking s ∈ S with ⟨γ, α_{s}⟩ < 0 and putting γ_{1} = s·γ, we have γ_{1} ≠ γ and γ_{1} −γ ∈ ℝ_{≥0}Π. Iterating, we obtain an infinite sequence γ_{0} = γ, γ_{1}, γ_{2}, … of distinct positive roots in W · γ inductively, proving the claim.

We also prepare a technical lemma.

Lemma 2.11. Let β, γ ∈ Φ^{+}, I ⊆ S and suppose that supp(γ) ⊈ supp(β) and supp(γ) ⊈ I. Then s_{γ} ∉ s_{β}W_{I}.

Proof. Assume contrary that s_{γ} = s_{β}w for some w ∈ W_{I}. Then we have w · γ = s_{β}s_{γ} · γ = −s_{β} · γ, while w · γ ∈ Φ^{+} and s_{β} · γ ∈ Φ^{+} by the hypothesis. This is a contradiction.

Finite, affine and hyperbolic Coxeter groups

The finite irreducible Coxeter groups are completely classified, as summarized in [9, Chapter 2]. If I ⊆ S and W_{I} is finite, let w_{0}(I) denote the unique longest element of W_{I}, which is an involution and maps Π_{I} onto −Π_{I}. If W_{I} is irreducible (but not necessarily finite) and 1 ≠ w ∈ W_{I}, then we have I^{w} = I if and only if W_{I} is finite and w = w_{0}(I). This implies the well-known fact that the center Z(W_{I}) of an arbitrary W_{I} is an elementary abelian 2-group. Moreover, if W_{I} is finite but not irreducible, then w_{0}(I) = w_{0}(I_{1}) ⋯ w_{0}(I_{k}) where W_{I_{1}}, …, W_{I_{k}} are the irreducible components of W_{I}. It is well known that, if w ∈ W_{I} and ℓ(ws) < ℓ(w) for all s ∈ I, then W_{I} is finite and w = w_{0}(I).

Theorem 2.12 ([17, Theorem A]). For any involution w ∈ W, there is a finite W_{I} (where I ⊆ S) such that w is conjugate to w_{0}(I) and w_{0}(I) ∈ Z(W_{I}).

The cases where |W_{I}| < ∞ and w_{0}(I) ∈ Z(W_{I}) are determined as well. Let W be an irreducible Coxeter group of finite rank. Then W is called affine

or compact hyperbolic, respectively, if the bilinear form ⟨ , ⟩ satisfies that (1) it is positive semidefinite or nondegenerate, respectively; (2) it is not positive definite; and (3) its restriction to any proper subspace V_{I} ⊂ V (where I ⊂ S) is positive definite. (See [9, Section 6.8] for another definition of compact hyperbolicness and its equivalence to ours.) The next proposition says that these are the minimal non-finite irreducible Coxeter groups.

Proposition 2.13. Let W be a Coxeter group of finite rank.

1. ([9, Theorem 6.4]) We have |W | < ∞ if and only if ⟨ , ⟩ is positive definite.

2. If |W | = ∞ and every proper standard parabolic subgroup W_{I} ⊂ W is finite, then W is irreducible, and is either affine or compact hyperbolic.

Proof. For (2), it is easy to show that this W is irreducible. Thus by (1) and the definition of compact hyperbolicness, it now suffices to show that this ⟨ , ⟩ is positive semidefinite if it is degenerate. This follows from the observation that now V is the sum of a positive definite subspace V_{S∖{s}} (where s ∈ S; see (1)) of codimension 1 and the nonzero radical V^{⊥} of V (note that V^{⊥} ⊈ V_{S∖{s}}).

The affine and the compact hyperbolic Coxeter groups are completely determined in [9, Chapter 2 and Section 6.9]. See the lists in Figures 1 and 2, where we abbreviate s_{i} to i. Note that the names of the compact hyperbolic Coxeter groups given here are not standard and are very temporary.

On the other hand, it is shown in [15, Proposition 4.14] that the infinite irreducible Coxeter groups of infinite ranks, in which every proper standard parabolic subgroup of finite rank is finite, are exhausted by Figure 3.

On centralizers and normalizers in Coxeter groups

The centralizers and the normalizers in Coxeter groups play important roles in our arguments. Here we summarize some properties which we require.

Lemma 2.14 (e.g. [15, Lemma 4.4]). Let W_{I} be a finite standard parabolic subgroup of W such that w_{0}(I) ∈ Z(W_{I}). Then the centralizer Z_{W} (w_{0}(I)) coincides with the normalizer N_{W} (W_{I}) of W_{I} in W.

Proposition 2.15. Let W be an infinite irreducible Coxeter group. Then no involution in W is almost central in W (see Section 2.1 for terminology).

Proof. First, if s ∈ S, then W acts transitively on the conjugacy class of s in W, which is an infinite set (Lemma 2.8), so the kernel of this action is Z_{W} (s) and has infinite index in W. Thus s is not almost central.

By Theorem 2.12, it suffices to prove that the longest element w_{0}(I) of any finite W_{I} ≠ 1 with w_{0}(I) ∈ Z(W_{I}) is not almost central. Note that Z_{W} (w_{0}(I)) = N_{W} (W_{I}) (Lemma 2.14), while [N_{W} (W_{I}) : Z_{W} (W_{I})] < ∞ since |W_{I}| < ∞. By the first paragraph, Z_{W} (s) has infinite index in W for any s ∈ I, so do Z_{W} (W_{I}) (see (2.1)) and Z_{W} (w_{0}(I)), as desired.

Finally, in [15, Theorem 3.1], the centralizer of a normal subgroup generated by involutions in an irreducible W is completely determined. The following observation is an easy consequence of the result.

Proposition 2.16 (See [15, Theorem 3.1]). Suppose that W is an arbitrary Coxeter group, and H ≤ W is a subgroup generated by involutions which is normal in W. Then Z_{W} (H) is also generated by involutions.

3 The main theorem and its applications

The first subsection of this section summarizes the main theorem of this paper (Theorem 3.1) and its corollary (Theorem 3.2) together with some notational remarks. The second subsection consists of some examples, and explains what our theorem yields in these cases. Finally, the third subsection is devoted to an application of our theorem to the analysis of the isomorphism problem of Coxeter groups (the problem of deciding which Coxeter groups are isomorphic as abstract groups), which is the original motive of this research.

[figure: Ã_{n} (n ≥ 2); ⋯; 1; 2; n − 1; n; n + 1; Ã_{1}; ∞; 1; 2; B̃_{n} (n ≥ 3); ⋯; 4; 1; 2; 3; 4; n; n + 1; C̃_{n} (n ≥ 2); ⋯; 4; 4; 1; 2; 3; n − 1; n; n + 1; D̃_{n} (n ≥ 4); ⋯; 1; 2; 3; 4; n − 2; n − 1; n; n + 1; Ẽ_{6}; 1; 2; 3; 4; 5; 6; 7; Ẽ_{7}; 1; 2; 3; 4; 5; 6; 7; 8; Ẽ_{8}; 1; 2; 3; 4; 5; 6; 7; 8; 9; F̃_{4}; 4; 1; 2; 3; 4; 5; G̃_{2}; 6; 1; 2; 3]

Figure 1: List of the affine Coxeter groups

[figure: X_{1}; 4; 1; 2; 3; 4; 5; X_{2}(m_{1}, m_{2}); (  ); 3 ≤ m_{1}, m_{2} ≤ 5; (m_{1}, m_{2}) ≠ (3, 3); m_{1}; m_{2}; 1; 2; 3; 4; X_{3}(m_{1}, m_{2}, m_{3}); (  ); 3 ≤ m_{1}, m_{2}, m_{3} < ∞; (m_{1}, m_{2}, m_{3}) ≠ (3, 3, 3); m_{1}; m_{3}; m_{2}; 1; 2; 3; Y_{1}; 5; 1; 2; 3; 4; Y_{2}; 5; 1; 2; 3; 4; 5; Y_{3}(m); (3 ≤ m ≤ 5); 5; m; 1; 2; 3; 4; 5; Y_{4}(m); (m = 4, 5); 5; m; 1; 2; 3; 4; Y_{5}; 5; 1; 2; 3; 4; Y_{6}(m_{1}, m_{2}); (  ); 3 ≤ m_{2} ≤ m_{1} < ∞, 5 ≤ m_{1}; (m_{1}, m_{2}) ≠ (5, 3), (6, 3); m_{1}; m_{2}; 1; 2; 3]

Figure 2: List of the compact hyperbolic Coxeter groups

[figure: A_{∞}; ⋯; 1; 2; 3; A_{±∞}; ⋯; ⋯; −2; −1; 0; 1; 2; B_{∞}; ⋯; 4; 1; 2; 3; 4; D_{∞}; ⋯; 1; 2; 3; 4; 5]

Figure 3: Some Coxeter groups of infinite ranks

3.1 Main theorem

First we prepare some notations. Let W be an arbitrary Coxeter group, and G a group acting on W via a map ρ : G → Aut Γ, g ↦ ρ_{g}, yielding the semidirect product W ⋊ G with respect to ρ. Let C_{W}^{fin} and C_{W}^{inf} be the set of the finite and the infinite irreducible components of W, respectively, and C_{W} = C_{W}^{fin} ∪ C_{W}^{inf}. Then the G-action permutes the elements of each of C_{W}, C_{W}^{fin} and C_{W}^{inf}. Let ρ^{†} : G → Sym(C_{W}^{fin}), g ↦ ρ_{g}^{†}, denote the induced permutation representation of G on C_{W}^{fin}. For C ⊆ C_{W}, let W (C) be the product of the irreducible components in C, and put W_{fin} = W (C_{W}^{fin}) and W_{inf} = W (C_{W}^{inf}). Moreover, for an arbitrary group H, let H_{ACI} be the set of the almost central involutions in H (see Section 2.1 for the terminology).

Now our main theorem is as follows:

Theorem 3.1. Here we adopt the above notations.

1. Let wg be an involution in W ⋊ G with w ∈ W and g ∈ G. Then wg is almost central in W ⋊ G if and only if w ∈ W (O_{ρ}) and g ∈ G_{ρ} ∪ {1}, where G_{ρ} is the set of all h ∈ G_{ACI} satisfying the following condition:

ρ_{h} is identity on all irreducible components of W

except a finite number of finite irreducible components, (3.1)

and O_{ρ} ⊆ C_{W}^{fin} is the union of the ρ^{†}(G)-orbits with finite cardinalities.

2. We have

⟨(W ⋊ G)_{ACI}⟩ = W (O_{ρ}) ⋊ ⟨G_{ρ}⟩.

Note that, assuming Theorem 5.1 below, the condition (3.1) is equivalent to the finiteness of the index [W : W^{ρ_{g}}] of the fixed-point subgroup W^{ρ_{g}} by ρ_{g}. The proof of Theorem 3.1 is postponed until Section 6.

Since the subgroup ⟨H_{ACI}⟩ of a group H is determined by the isomorphism type of H only, we obtain the following consequence.

Theorem 3.2. For i = 1, 2, let W_{i} ⋊ G_{i} be a semidirect product (via ρ_{i} : G_{i} → Aut Γ_{i}) as in Theorem 3.1, and f : W_{1} ⋊ G_{1} →^{∼} W_{2} ⋊ G_{2} a group isomorphism. Then f maps W_{1}(O_{ρ_{1}}) ⋊ (G_{1})_{ρ_{1}} onto W_{2}(O_{ρ_{2}}) ⋊ (G_{2})_{ρ_{2}}.

3.2 Examples

First we observe that, if |G| < ∞, then every ρ^{†}(G)-orbit in C_{W}^{fin} is finite, so O_{ρ} = C_{W}^{fin} in Theorem 3.1, therefore ⟨(W ⋊ G)_{ACI}⟩ = W_{fin} ⋊ G_{ρ} and G_{ρ} is generated by all involutions h ∈ G satisfying (3.1).

Example 3.3. Let W be an arbitrary Coxeter group. Then, by putting G = 1, Theorem 3.1 shows that ⟨W_{ACI}⟩ = W_{fin}. Thus if f : W →^{∼} W′ is a group isomorphism between two Coxeter groups, then f(W_{fin}) = W′_{fin}; hence, by taking W′ = W and f = id_{W}, it follows that the factor W_{fin} is independent on the choice of the generating set S ⊆ W.

Example 3.3 is slightly generalized as follows:

Example 3.4. Let W wr S_{n} = W^{n} ⋊ S_{n} denote the wreath product of W with the symmetric group S_{n} on n letters, so σ ∈ S_{n} acts on (w_{1}, …, w_{n}) ∈ W^{n} by ρ_{σ}(w_{1}, …, w_{n}) = (w_{σ^{−1}(1)}, …, w_{σ^{−1}(n)}). Then Theorem 3.1 implies that

⟨(W wr S_{n})_{ACI}⟩ = \begin{cases}W wr S_{n} & if |W | < ∞; \\ W_{fin}^{n} & if |W | = ∞. \end{cases}

Indeed, if |W | = ∞, then W possesses either an infinite irreducible component or infinitely many finite irreducible components, so no non-identity σ ∈ S_{n} satisfies the condition (3.1) in any case.

We say that an irreducible component W_{I} of W has finite multiplicity in W if W possesses only finitely many irreducible components with Coxeter graph isomorphic to Γ_{I}. Note that, even if |G| = ∞, the factor W (O_{ρ}) in the theorem contains all W_{I} ∈ C_{W}^{fin} with finite multiplicities.

Example 3.5. Let G = Aut Γ ∗ Aut Γ be the free product of two copies of Aut Γ, and ρ : G → Aut Γ the map obtained by forgetting the distinction of the two factors Aut Γ of G. Then O_{ρ} is the set of all W_{I} ∈ C_{W}^{fin} with finite multiplicities, and G_{ρ} = 1 since we have G_{ACI} = ∅ by properties of free products. Roughly speaking, Theorem 3.1 extracts the finite irreducible components of W with finite multiplicities in this manner.

For the final example, we prepare some further facts and notations. Let Γ^{odd} denote the odd Coxeter graph of a Coxeter group W, which is the subgraph of Γ obtained by removing all the edges with non-odd labels. It is well known (see [9, Exercise 5.3]) that two orbits W · α_{s} and W · α_{t} (where s, t ∈ S) intersects nontrivially if and only if s and t lie in the same connected component of Γ^{odd}. Let S = S_{1} ⊔ S_{2} be a partition where both factors are unions of connected components of Γ^{odd}, and Φ_{∼S_{1}} = ⋃_{s∈S_{1}} W · α_{s} ⊆ Φ. Now a general theorem of Vinay V. Deodhar [5] or of Matthew Dyer [6] shows that the subgroup W (Φ_{∼S_{1}}) generated by the reflections s_{γ} along γ ∈ Φ_{∼S_{1}}, which is normal in W since Φ_{∼S_{1}} is W -invariant, is a Coxeter group. Moreover, the set Φ_{∼S_{1}} plays the role of a root system of W (Φ_{∼S_{1}}); for example, any non-identity w ∈ W (Φ_{∼S_{1}}) sends some γ ∈ Φ_{∼S_{1}}^{+} to a negative root.

Now we show that W decomposes as W (Φ_{∼S_{1}}) ⋊ W_{S_{2}}. First, if 1 ≠ w ∈ W (Φ_{∼S_{1}}) ∩ W_{S_{2}}, then w · γ ∈ Φ^{−} for some γ ∈ Φ_{∼S_{1}}^{+} as mentioned above, and γ ∈ Φ_{S_{2}} since w ∈ W_{S_{2}}. Now by (2.6), we have γ ∈ W · α_{s} ∩ W · α_{t} for some s ∈ S_{1} and t ∈ S_{2}, contradicting the choice of the partition S = S_{1} ⊔ S_{2}. Thus we have W (Φ_{∼S_{1}}) ∩ W_{S_{2}} = 1, while S ⊆ W (Φ_{∼S_{1}})W_{S_{2}} generates W, yielding the desired decomposition.

Moreover, this argument also shows that each s ∈ S_{2} preserves the set Φ_{∼S_{1}}^{+} of positive roots of W (Φ_{∼S_{1}}) (since α_{s} ∉ Φ_{∼S_{1}}), so also the set of simple roots of W (Φ_{∼S_{1}}), therefore W_{S_{2}} acts on W (Φ_{∼S_{1}}) as graph automorphisms. Thus Theorem 3.1 yields the following observation:

Example 3.6. In the situation, suppose further that W is infinite and irreducible. Then ⟨W_{ACI}⟩ = 1 (Example 3.3), while W (Φ_{∼S_{1}})(O_{ρ}) contains all the finite irreducible components of W (Φ_{∼S_{1}}) with finite multiplicities as mentioned above. Since 1 = W (Φ_{∼S_{1}})(O_{ρ}) ⋊ (W_{S_{2}})_{ρ} (Theorem 3.2), it follows that no finite irreducible component of W (Φ_{∼S_{1}}) has finite multiplicity in W (Φ_{∼S_{1}}).

In addition, if W_{S_{2}} is finite, then we have W (Φ_{∼S_{1}})(O_{ρ}) = W (Φ_{∼S_{1}})_{fin}. Now it follows that W (Φ_{∼S_{1}}) possesses no finite irreducible component.

3.3 Application to the isomorphism problem of Coxeter groups

An important phase of the isomorphism problem of Coxeter groups is of deciding whether a given group isomorphism f : W →^{∼} W′ between two Coxeter groups W and W′ (with generating sets S and S′, respectively) maps the set S^{W} of reflections in W onto that S′^{W′} in W′; or, whether the subset S^{W} of W is independent on the choice of S. Note that, as is shown in [2, Lemma 3.7], we have f(S^{W}) = S′^{W′} if and only if f(S) ⊆ S′^{W′}. Roughly speaking, our result below measures how f(s) differs from reflections for each s ∈ S, within a certain compass. In most successful cases, the result is able to show that all f(s) are reflections in W′ (see Theorem 3.7).

Note that our results cover the case |S| = ∞ as well, in contrast with almost all of the preceding results on the isomorphism problem which cover the case of finite ranks only.

Preliminaries on centralizers and normalizers

The central tools of our argument are the centralizers Z_{W} (W_{I}) and the normalizers N_{W} (W_{I}) of standard parabolic subgroups W_{I}, which are described by the author [14] in a general setting (note that the normalizers had already been described by Brigitte Brink and Robert B. Howlett [3]). Here we summarize some of the author’s results which we use.

Here we require the result only for the case that |W_{I}| < ∞ and w_{0}(I) ∈ Z(W_{I}). Now Z_{W} (W_{I}) and N_{W} (W_{I}) admit the following decompositions:

Z_{W} (W_{I}) = (Z(W_{I}) × W^{⊥I}) ⋊ Y_{I} and N_{W} (W_{I}) = (W_{I} × W^{⊥I}) ⋊ Ỹ_{I}. (3.2)

Here W^{⊥I} denotes the subgroup of W generated by the reflections in the set Z_{W} (W_{I}) ∖ W_{I}, which is a Coxeter group by a theorem of Deodhar [5] or of Dyer [6]. Since Z(W_{I}) is an elementary abelian 2-group, both Z(W_{I}) × W^{⊥I} and W_{I} × W^{⊥I} are also Coxeter groups. The factor Ỹ_{I} of N_{W} (W_{I}) acts on W_{I} × W^{⊥I} as graph automorphisms, preserving the factor W_{I}. The factor Y_{I} of Z_{W} (W_{I}) is torsion-free and is the kernel of the induced action of Ỹ_{I} on W_{I}, so Y_{I} is normal and has finite index in Ỹ_{I} since |W_{I}| < ∞.

The results

Let f : W →^{∼} W′ be a group isomorphism between two Coxeter groups W and W′ as above, and I ⊆ S a subset with |W_{I}| < ∞ and w_{0}(I) ∈ Z(W_{I}). Our temporal subject is the element f(w_{0}(I)) ∈ W′. Since f(w_{0}(I)) is an involution in W′ as well as w_{0}(I), Theorem 2.12 allows us to assume for simplicity that f(w_{0}(I)) = w_{0}(J) for some J ⊆ S′ with |W′_{J}| < ∞ and w_{0}(J) ∈ Z(W′_{J}). Let Y′_{J} and Ỹ′_{J} denote the last factors of Z_{W′}(W′_{J}) and of N_{W′}(W′_{J}), respectively (see (3.2)).

We start with a very simple observation: since the isomorphism f maps w_{0}(I) to w_{0}(J), it also maps Z_{W} (w_{0}(I)) onto Z_{W′}(w_{0}(J)), so the combination of Lemma 2.14 and (3.2) yields the following isomorphism

f : (W_{I} × W^{⊥I}) ⋊ Ỹ_{I} ⟶^{∼} (W′_{J} × W′^{⊥J}) ⋊ Ỹ′_{J}.  (3.3)

Let ρ and ρ′ denote the maps representing the actions of Ỹ_{I} and Ỹ′_{J} in (3.3), respectively. Then by (3.3) and the results in Section 3.3, Theorem 3.2 yields the following isomorphism

f : (W_{I} × W^{⊥I})(O_{ρ}) ⋊ (Ỹ_{I})_{ρ} ⟶^{∼} (W′_{J} × W′^{⊥J})(O′_{ρ′}) ⋊ (Ỹ′_{J})_{ρ′}.  (3.4)

Now the left and the right sides of (3.4) contain, as normal subgroups, W_{I} and W′_{J} which are ρ(Ỹ_{I})-invariant and ρ′(Ỹ′_{J})-invariant, respectively. Thus if we know much enough of the structure of the left side of (3.4), then we would be able to say something about the variation of the set J, so about the property of f(w_{0}(I)). This is hopeful at least for individual cases, since [14] also gives a method for computing the explicit structure of the decompositions (3.2).

From now, we assume further that Ỹ_{I} = Y_{I} (this is satisfied if W_{I} admits no nontrivial graph automorphism). For an arbitrary group G, let G_{INV} be the set of the involutions in G, so ⟨G_{INV}⟩ ⊴ G and ⟨G_{INV}⟩ is determined by the isomorphism type of G only as well as ⟨G_{ACI}⟩. Then, since both W_{I} × W^{⊥I} and W′_{J} ×W′^{⊥J} are generated by involutions and the torsion-free group Y_{I} possesses no involution, we can derive from (3.3) the following isomorphism

f : W_{I} × W^{⊥I} ⟶^{∼} (W′_{J} × W′^{⊥J}) ⋊ G, where G = ⟨(Ỹ′_{J})_{INV}⟩,  (3.5)

by taking the ⟨(∗)_{INV}⟩ of both sides. Now consider the centralizers of the normal subgroups f^{−1}(W′_{J}) and W′_{J} in the left and the right sides of (3.5), respectively, which are also isomorphic via f. Since f^{−1}(W′_{J}) is generated by involutions, Proposition 2.16 implies that the centralizer in the left side is also generated by involutions, so is the centralizer in the right side. The latter is the intersection of the right side of (3.5) and Z_{W′}(W′_{J}) = (Z(W′_{J}) × W′^{⊥J}) ⋊ Y′_{J}, that is

(Z(W′_{J}) × W′^{⊥J}) ⋊ (Y′_{J} ∩ G),

and all of its involutions are contained in the former factor since Y′_{J} ∩ G is torsion-free as well as Y′_{J}. Thus it follows that Y′_{J} ∩ G = 1, so the G-action on the finite group W′_{J} is faithful, therefore G is also finite. Hence, as mentioned in the first paragraph of Section 3.2, (3.5) and Theorem 3.2 yield the following isomorphism

f : W_{I} × W^{⊥I}_{fin} ⟶^{∼} (W′_{J} × W′^{⊥J}_{fin}) ⋊ G_{ρ′}.  (3.6)

This reduces our problem to the study of semidirect product decompositions of Coxeter groups whose irreducible components are finite.

Finally, specializing to the case I = {s}, we obtain the following result.

Theorem 3.7. Let (W, S) be an arbitrary Coxeter system.

1. Suppose that s ∈ S, and W^{⊥s}_{fin} is either trivial or generated by a single reflection conjugate to s. Then f(s) ∈ S′^{W′} for any Coxeter system (W′, S′) and any group isomorphism f : W →^{∼} W′.

2. Suppose that every s ∈ S satisfies the hypothesis of (1). Then f(S) ⊆ S′^{W′} for any Coxeter system (W′, S′) and any group isomorphism f : W →^{∼} W′, so f preserves the set of reflections. Hence the set S^{W} is determined by W only and independent on the choice of S ⊆ W.

Proof. We only prove (1), since (2) follows immediately from (1) and the first remark of Section 3.3. Now the above argument works for I = {s}, so it suffices to deduce that |J| = 1, implying that f(s) = w_{0}(J) ∈ S′ as desired. This is immediately done if W^{⊥s}_{fin} = 1, since J ≠ ∅ and now both sides of (3.6) have cardinality 2.

Suppose that W^{⊥s}_{fin} = ⟨t⟩ with t ∈ W conjugate to s. Then both sides of (3.6) have cardinality 4. Thus if |J| ≠ 1, then it follows that J = {s′, t′} for two commuting generators s′, t′ ∈ S′ and the right side of (3.6) is W′_{J} itself, so we have an isomorphism f : ⟨s⟩ × ⟨t⟩ →^{∼} ⟨s′⟩ × ⟨t′⟩. Since we assumed that f(s) = w_{0}(J) = s′t′, it follows that f(t) is either s′ or t′, which cannot be conjugate to f(s) = s′t′ in W′, contradicting the choice of t. Hence |J| = 1.

Moreover, a forcecoming paper [13] of the author will describe for which s ∈ S the hypothesis is indeed satisfied, and show that this case occurs very frequently.

4 Essential elements and Coxeter elements

Krammer introduced in his Ph.D. thesis [10] the notion of essential elements of Coxeter groups. An element w of a Coxeter group W is called essential in W if the parabolic closure P(w) of w is W itself (see Section 2.2 for terminology). Note that any W of infinite rank cannot possess an essential element, while a Coxeter element s_{1}s_{2} ⋯ s_{n} of an infinite irreducible W of finite rank (where S = {s_{1}, s_{2}, …, s_{n}}) is always essential in W (see Theorem 4.1). Here we summarize some properties of essential elements required in later sections, as follows:

Theorem 4.1. Let W be an infinite irreducible Coxeter group of finite rank.

1. Any essential element of W has infinite order.

2. Let 0 ≠ k ∈ ℤ. Then w ∈ W is essential in W if and only if w^{k} is essential in W.

3. If n = |S| and γ_{1}, …, γ_{n} ∈ Φ are linearly independent, then s_{γ_{1}} ⋯ s_{γ_{n}} is essential in W. Hence any Coxeter element of W is essential in W.

The claim (1) is an immediate consequence of a well-known theorem of Jacques Tits, which says that any finite subgroup of a Coxeter group is contained in a finite parabolic subgroup (see e.g. [1, Lemma 1.2] for a proof). On the other hand, (2) and (3) are shown by Paris in his recent preprint [16]; however, he proved (3) only for Coxeter elements though his idea is adaptable applicable to the generalized version. Here we include proofs of (2) and (3) along Paris’ idea for the sake of completeness.

For (2), we fix W and w as in the statement. For γ ∈ Φ, let σ_{γ}^{w} = ((σ_{γ}^{w})_{n})_{n∈ℤ} be the infinite sequence of + and − such that (σ_{γ}^{w})_{n} = ε if and only if w^{n}·γ ∈ Φ^{ε}. We define (σ_{γ}^{w})_{∞} (or (σ_{γ}^{w})_{−∞}, respectively) to be ε ∈ {+, −} if (σ_{γ}^{w})_{n} = ε (or (σ_{γ}^{w})_{−n} = ε, respectively) for all sufficiently large n. Following [10], we say that γ is w-periodic if w^{n} · γ = γ for some n ≠ 0. Now we include the proofs of the following two lemmas for the sake of completeness.

Lemma 4.2 (See [10, Proposition 5.2.2]). If γ ∈ Φ is not w-periodic, then only finitely many sign-changes occur in the sequence σ_{γ}^{w}.

Proof. By the hypothesis, all roots w^{n}·γ such that (σ_{γ}^{w})_{n} ≠ (σ_{γ}^{w})_{n+1} are distinct and contained in the finite set Φ [w] ∪ −Φ [w].

A root γ ∈ Φ is called w-odd (see [10]) if it is not w-periodic (so both (σ_{γ}^{w})_{±∞} are defined; see Lemma 4.2) and (σ_{γ}^{w})_{∞} ≠ (σ_{γ}^{w})_{−∞}. A reflection s_{γ} is called w-odd if γ is w-odd.

Lemma 4.3 (See [10, Lemma 5.2.7]). For k ∈ ℤ∖{0}, a root of W is w-odd if and only if it is w^{k}-odd.

Proof. Note that γ ∈ Φ is w-periodic if and only if it is w^{k}-periodic. Thus for a non-w-periodic γ, all of (σ_{γ}^{w})_{±∞} and (σ_{γ}^{w^{k}})_{±∞} are defined (Lemma 4.2) and we have

(σ_{γ}^{w^{k}})_{±∞} = \begin{cases}(σ_{γ}^{w})_{±∞} & if k > 0; \\ (σ_{γ}^{w})_{∓∞} & if k < 0, \end{cases}

respectively. Thus the claim follows.

Let P^{∞}(w) denote the subgroup of W generated by the w-odd reflections. The following result of [10] is crucial in our argument.

Proposition 4.4 (See [10, Corollary 5.8.7]). The parabolic closure P(w) is a direct product of P^{∞}(w) and a finite number of finite groups.

Moreover, the following result of the author [15] is also required. See also [16, Theorem 4.1] for the case of finite ranks.

Proposition 4.5 ([15, Theorem 3.3]). If W is an infinite irreducible Coxeter group, then W is directly indecomposable as an abstract group.

Corollary 4.6. Suppose that W is infinite and irreducible. Then w ∈ W is essential in W if and only if P^{∞}(w) = W.

Proof. The ‘if’ part is a consequence of Proposition 4.4. For the “only if” part, assume that P(w) = W. Then Proposition 4.4 implies that W is the direct product of P^{∞}(w) and certain finite groups, while W is directly indecomposable (Proposition 4.5). Thus W must coincide with one of the direct factors, which cannot be finite since |W | = ∞, so W = P^{∞}(w) as desired.

Now the claim (2) of Theorem 4.1 follows easily from Lemma 4.3 and Corollary 4.6, since the w^{k}-odd reflections are precisely the w-odd reflections.

For the proof of (3), we prepare two lemmas. Here we say that (W, S) is (non)degenerate to signify the (non)degenerateness of the bilinear form ⟨ , ⟩, respectively.

Lemma 4.7 (See [16, Lemma 3.2]). Let W be a Coxeter group of finite rank. Then there is a nondegenerate Coxeter system (W̃, S̃) of finite rank such that S ⊆ S̃ and W̃_{S} = W_{S}.

Proof. We put n = |S| and S = {s_{1}, s_{3}, …, s_{2n−1}}, and apply the following algorithm inductively for 1 ≤ k ≤ n, beginning with S̃ = S: if the Coxeter system (⟨I_{k}⟩, I_{k}) (where I_{k} = {s_{i} ∈ S̃ | i ≤ 2k}) is degenerate, add a new generator s_{2k} to S̃ so that s_{2k−1}s_{2k} has infinite order and s_{2k} commutes with the other elements of S̃.

By computing the determinant of the matrix of the bilinear form with respect to the basis {α_{s_{i}}}_{i}, it is checked inductively that the Coxeter system (⟨I_{k}⟩, I_{k}) will be nondegenerate when the k-th step is done. Hence the Coxeter system (W̃, S̃) = (⟨I_{n}⟩, I_{n}) obtained finally is the desired one.

Lemma 4.8. Any element of a proper standard parabolic subgroup W_{I} of W has a nonzero 1-eigenvector in V.

Proof. It suffices to consider the case that S = {s_{1}, s_{2}, …, s_{n}} is finite and I = S∖{s_{n}}. Then, by definition of the W -action, the n-th row of the representation matrix A_{w} of w ∈ W_{I} relative to the basis Π of V is (0 0 ⋯ 0 1). Thus the matrix I_{n} − A_{w} is singular as desired.

The following property is the essence of the claim (3) of Theorem 4.1.

Proposition 4.9. Let W be a Coxeter group with |S| = n < ∞, and suppose that γ_{1}, …, γ_{n} ∈ Φ are linearly independent. Then the standard parabolic closure of s_{γ_{1}} ⋯ s_{γ_{n}} ∈ W is W itself.

Proof. Assume contrary that w = s_{γ_{1}} ⋯ s_{γ_{n}} ∈ W_{I} for a proper W_{I} ⊂ W. We may assume without loss of generality that (W, S) is nondegenerate, since we can extend S to S̃ = S ⊔ {t_{1}, …, t_{m}} as in Lemma 4.7 and consider t_{1} ⋯ t_{m}w ∈ W̃_{J} instead of w, where J = S̃ ∖ (S ∖ I). Choose a nonzero v ∈ V such that w · v = v (Lemma 4.8). Then, since (W, S) is nondegenerate, there is an index i such that ⟨v, γ_{i}⟩ ≠ 0 and ⟨v, γ_{j}⟩ = 0 for all j > i. This implies that w · v = s_{γ_{1}} ⋯ s_{γ_{i}} · v, which is the sum of s_{γ_{i}} · v = v − 2⟨v, γ_{i}⟩γ_{i} and a linear combination of γ_{1}, …, γ_{i−1}. Now the property w · v = v yields an expression of 2⟨v, γ_{i}⟩γ_{i} as a linear combination of the other γ_{j}, contradicting the linear independence of γ_{1}, …, γ_{n}. Hence the claim follows.

Now the claim (3) of Theorem 4.1 is easily proved, since the hypothesis of Proposition 4.9 is invariant under the action of W. Hence the proof of Theorem 4.1 is concluded.

5 On the fixed-point subgroups by Coxeter graph automorphisms

The subject of this section is the fixed-point subgroup

W^{τ} = {w ∈ W | τ(w) = w}

of a Coxeter group W by a graph automorphism τ ∈ Aut Γ (as mentioned in Section 2.2, the automorphism of W induced by τ is also denoted by τ). Let τ\S denote the set of the ⟨τ⟩-orbits in S. Then it was shown by Steinberg [18, Theorem 1.32] that W^{τ} is a Coxeter group with respect to the following generating set

S(W^{τ}) = {w_{0}(I) ∈ W | I ∈ τ\S and |W_{I}| < ∞}

(see also [11] and [12]). Here we show the following properties of the subgroup W^{τ}, which will be used in the proof of the main theorem.

Theorem 5.1. Let W be an arbitrary Coxeter group and τ ∈ Aut Γ. Then W^{τ} has finite index in W if and only if τ is identity on all irreducible components of W except a finite number of finite irreducible components.

Theorem 5.2. Let W be an infinite irreducible Coxeter group and τ ∈ Aut Γ.

1. If |W_{I}| < ∞ for all I ∈ τ\S, then the Coxeter group W^{τ} is also infinite and irreducible with respect to the generating set S(W^{τ}).

2. Suppose that the hypothesis of (1) fails and every orbit I ∈ τ\S is finite. Then for any 1 ≠ w ∈ W^{τ}, there is an element u ∈ W of infinite order such that u^{k}wτ(u)^{−k} ≠ w for all 0 ≠ k ∈ ℤ.

Note that the result on infiniteness of W^{τ} in Theorem 5.2 (1) is mentioned in [11, Section 5] without proof in a generalized setting.

5.1 Proof of Theorem 5.1

Our first step is to prove the following lemma:

Lemma 5.3. Let W be an (irreducible) affine or compact hyperbolic Coxeter group with type W ≠ Ã_{1} (see Section 2.2 for terminology). Suppose further that Aut Γ ≠ {id_{S}}. Then for any id_{S} ≠ τ ∈ Aut Γ, there is an element w ∈ W of infinite order such that ⟨w⟩ ∩ ⟨τ(w)⟩ = 1.

From now until the end of the proof of Lemma 5.3, we assume that S is finite and the base field of the (finite-dimensional) geometric representation space V is extended from ℝ to ℂ. Then the bilinear form ⟨ , ⟩ and the faithful W -action also extend naturally so that W is embedded injectively in the group of orthogonal linear transformations of V relative to ⟨ , ⟩. For λ ∈ ℂ, let V_{λ}(w) denote the λ-eigenspace of w ∈ W, and let V\sqrt{1}(w), V_{≠\sqrt{1}}(w) be the sum of V_{λ}(w) where λ runs over the roots of unity, over ℂ ∖ {0} except the roots of unity, respectively. Then some elementary linear algebra shows that, if w ∈ W, 0 ≠ λ ∈ ℂ and 0 ≠ k ∈ ℤ, then V_{λ}(w^{k}) is the sum of V_{μ}(w) where μ ∈ ℂ varies subject to μ^{k} = λ. Hence we have V\sqrt{1}(w^{k}) = V\sqrt{1}(w) and V_{≠\sqrt{1}}(w^{k}) = V_{≠\sqrt{1}}(w) whenever k ≠ 0.

Now we have the following:

Lemma 5.4. Let w_{1}, w_{2} ∈ W and suppose that either V\sqrt{1}(w_{1}) ≠ V\sqrt{1}(w_{2}) or V_{≠\sqrt{1}}(w_{1}) ≠ V_{≠\sqrt{1}}(w_{2}). Then ⟨w_{1}⟩ ∩ ⟨w_{2}⟩ = 1.

Proof. Assume contrary that k, ℓ ∈ ℤ ∖ {0} and w_{1}^{k} = w_{2}^{ℓ}. Then, in the first case V\sqrt{1}(w_{1}) ≠ V\sqrt{1}(w_{2}), the above observation implies that

V\sqrt{1}(w_{1}^{k}) = V\sqrt{1}(w_{1}) ≠ V\sqrt{1}(w_{2}) = V\sqrt{1}(w_{2}^{ℓ}),

contradicting the assumption w_{1}^{k} = w_{2}^{ℓ}. The other case is similar.

Define actions of τ ∈ Aut Γ on V and the dual space V^{∗} with dual basis {α_{s}^{∗} | s ∈ S} (as linear transformations) by

τ(α_{s}) = α_{τ(s)} and τ(α_{s}^{∗}) = α_{τ(s)}^{∗} for s ∈ S.

Then τ preserves the bilinear form ⟨ , ⟩, and we have τ(w) · τ(v) = τ(w · v) for w ∈ W and v ∈ V. Thus for 0 ≠ λ ∈ ℂ and w ∈ W, it follows that V_{λ}(τ(w)) = τ(V_{λ}(w)), V\sqrt{1}(τ(w)) = τ(V\sqrt{1}(w)) and V_{≠\sqrt{1}}(τ(w)) = τ(V_{≠\sqrt{1}}(w)). Moreover, we have τ(η)(τ(v)) = η(v) for η ∈ V^{∗} and v ∈ V. Note also that Ann(τ(V′)) = τ(Ann(V′)) for any subspace V′ ⊆ V, where Ann(V′) = {η ∈ V^{∗} | η(V′) = 0} denotes the annihilator of V′.

By these observations, we have the following lemmas. In these lemmas, write v^{⊥} = {v′ ∈ V | ⟨v, v′⟩ = 0} for v ∈ V.

Lemma 5.5. Let id_{S} ≠ τ ∈ Aut Γ, β, γ ∈ Φ^{+} and V′ ⊂ V a subspace of codimension 1. Suppose that ⟨β, γ⟩ = −1, V′ ⊆ β^{⊥} ∩ γ^{⊥} and Ann(V′) is not τ-invariant. Then w = s_{β}s_{γ} ∈ W has infinite order and ⟨w⟩ ∩ ⟨τ(w)⟩ = 1.

Proof. Since ⟨β, γ⟩ = −1, we have w^{k} · β = (2k + 1)β + 2kγ ≠ β for all k ≥ 1, showing that w has infinite order. Thus V\sqrt{1}(w) ≠ V, since otherwise we have V_{1}(w^{k}) = V and w^{k} = 1 for a sufficiently large k, a contradiction. Now we have

V′ ⊆ β^{⊥} ∩ γ^{⊥} ⊆ V_{1}(w) ⊆ V\sqrt{1}(w) ⊂ V and dim V − dim V′ = 1,

implying that V′ = V\sqrt{1}(w). Since Ann(V′) is not τ-invariant, we have

Ann(V\sqrt{1}(w)) ≠ τ(Ann(V\sqrt{1}(w))) = Ann(V\sqrt{1}(τ(w))),

so V\sqrt{1}(w) ≠ V\sqrt{1}(τ(w)). Hence Lemma 5.4 completes the proof.

Lemma 5.6. For i = 1, 2, let β_{i}, γ_{i} ∈ Φ^{+} and V^{(i)} ⊂ V a subspace of codimension 3, and suppose that ⟨β_{i}, γ_{i}⟩ < −1, V^{(i)} ⊆ β_{i}^{⊥} ∩ γ_{i}^{⊥} and ℂβ_{1} + ℂγ_{1} ≠ ℂβ_{2} + ℂγ_{2}. Then each w_{i} = s_{β_{i}}s_{γ_{i}} ∈ W has infinite order and ⟨w_{1}⟩ ∩ ⟨w_{2}⟩ = 1.

Proof. Put v_{i}^{±} = (−c_{i} ± \sqrt{c_{i}^{2} − 1})β_{i} + γ_{i} and λ_{i}^{±} = 2c_{i}^{2} − 1 ∓ 2c_{i}\sqrt{c_{i}^{2} − 1}, respectively, where c_{i} = ⟨β_{i}, γ_{i}⟩. Then a direct computation shows that w_{i} · v_{i}^{±} = λ_{i}^{±}v_{i}^{±} and |λ_{i}^{±}| ≠ 1, respectively, and λ_{i}^{+}λ_{i}^{−} = 1, so w_{i} has infinite order. Moreover, since β_{i}^{⊥} ∩ γ_{i}^{⊥} ⊆ V_{1}(w_{i}), the hypothesis implies that dim V − dim V_{1}(w_{i}) ≤ 3, so the characteristic polynomial χ_{w_{i}}(x) = det(x · id_{V} − w_{i}) of w_{i} decomposes as

χ_{w_{i}}(x) = (x − 1)^{|S|−3}(x − λ_{i}^{+})(x − λ_{i}^{−})(x − μ_{i}) where μ_{i} ∈ ℂ.

Now we have ±1 = det w_{i} = ±χ_{w_{i}}(0) = ±λ_{i}^{+}λ_{i}^{−}μ_{i} since w_{i} is a product of involutions, so μ_{i} = ±1. Thus V_{≠\sqrt{1}}(w_{i}) = ℂv_{i}^{+} + ℂv_{i}^{−} = ℂβ_{i} + ℂγ_{i}, so V_{≠\sqrt{1}}(w_{1}) ≠ V_{≠\sqrt{1}}(w_{2}) by the hypothesis. Hence Lemma 5.4 completes the proof.

Corollary 5.7. Let id_{S} ≠ τ ∈ Aut Γ, β, γ ∈ Φ^{+} and V′ ⊂ V a subspace of codimension 3. Suppose that ⟨β, γ⟩ < −1, V′ ⊆ β^{⊥} ∩ γ^{⊥} and ℂβ + ℂγ is not τ-invariant. Then w = s_{β}s_{γ} ∈ W has infinite order and ⟨w⟩ ∩ ⟨τ(w)⟩ = 1.

Proof. Note that τ(ℂβ + ℂγ) = ℂτ(β) + ℂτ(γ) and τ(s_{β}s_{γ}) = s_{τ(β)}s_{τ(γ)}. Then the claim follows from Lemma 5.6, where β_{1} = β, γ_{1} = γ, β_{2} = τ(β), γ_{2} = τ(γ), V^{(1)} = V′ and V^{(2)} = τ(V′).

Table 1: List for the proof of Lemma 5.3, affine case

W  τ  β  γ  Ann(V′)
Ã_{n} (n ≥ 2) τ(1) ≠ 1 α_{1}  011 ⋯ 11  2α_{1}^{∗} − α_{2}^{∗} − α_{n+1}^{∗}
B̃_{n} (n ≥ 3)  α_{1}  0122 ⋯ 22\sqrt{2}  2α_{1}^{∗} − α_{3}^{∗}
C̃_{n} (n ≥ 2)  α_{1} 0\sqrt{2}\sqrt{2} ⋯ \sqrt{2}\sqrt{2}1  \sqrt{2}α_{1}^{∗} − α_{2}^{∗}
D̃_{n} (n ≥ 4) τ(1) ≠ 1 α_{1}  0122 ⋯ 2211  2α_{1}^{∗} − α_{3}^{∗}
Ẽ_{6}  τ(1) ≠ 1 α_{1}  0232121  2α_{1}^{∗} − α_{2}^{∗}
Ẽ_{7}  α_{1}  02343212  2α_{1}^{∗} − α_{2}^{∗}

Table 2: List for the proof of Lemma 5.3, compact hyperbolic case

W  τ  β  γ  V′
X_{1}  α_{1} 01111 α_{3}, α_{4}
X_{2}(m_{1}, m_{2}) α_{4} s_{1}s_{2} · α_{3} α_{2}
X_{3}(m_{1}, m_{2}, m_{3}) τ(3) ≠ 3 α_{3} s_{2} · α_{1}  ∅
Y_{1}  α_{4}  α̃_{4}  α_{3}
Y_{2}  α_{5}  α̃_{5}  α_{1}, α_{2}
Y_{3}(5)  α_{5}  α̃_{5}  α_{1}, α_{2}
Y_{4}(5)  α_{4}  α̃_{4}  α_{1}
Y_{5}  α_{4}  α̃_{4}  α_{1}
Y_{6}(m, m) (m ≥ 5)  α_{3} s_{2} · α_{1}  ∅

Proof of Lemma 5.3. This lemma is deduced from Lemma 5.5 for affine case and Corollary 5.7 for compact hyperbolic case, by constructing the β, γ and V′ as in Tables 1 and 2 (see also Figures 1 and 2). Note that β + γ is the null root of W in an affine case. If |Aut Γ| ≥ 3, we assume by symmetry that τ satisfies the condition in the second column of the lists, where we abbreviate s_{i} to i. In the next two columns, a word c_{1}c_{2} ⋯ c_{r} (where r = |S|) signifies ∑_{i=1}^{r} c_{i}α_{i} ∈ V and α̃_{i} denotes the unique highest root of the finite Coxeter group W_{S∖{s_{i}}}. Finally, the last column gives a basis of V′ or of Ann(V′).

Now we cancel the assumption |S| < ∞ placed above. To prove Theorem 5.1, note that if τ ∈ Aut Γ leaves W_{I} ⊆ W invariant, then W_{I} possesses its own fixed-point subgroup W_{I}^{τ} which coincides with W^{τ} ∩ W_{I}.

Proof of Theorem 5.1. The only nontrivial part is the “only if” part, so we prove it. Note that, by (2.2), the hypothesis implies that

[G : W^{τ} ∩ G] < ∞ for any subgroup G ≤ W,  (5.1)

so W^{τ} ∩ G ≠ 1 for every infinite subgroup G of W.

Step 1: if I ⊆ S is finite, and W_{I} is infinite and irreducible, then τ(W_{I}) = W_{I}.

Assume contrary that τ(I) ≠ I, or equivalently I ⊈ τ(I). Then we have I ∩ τ(I) ≠ I, while W^{τ} ∩ W_{I} ⊆ W_{I} ∩ W_{τ(I)} = W_{I∩τ(I)} (see (2.4)), therefore no essential element in W_{I} lies in W^{τ}. Hence by Theorem 4.1, any power w^{k} (with k ≠ 0) of a Coxeter element w of W_{I} has infinite order and is not in W^{τ}, so we have W^{τ} ∩ ⟨w⟩ = 1, contradicting (5.1).

Step 2: the claim holds if W has finite rank.

Now it suffices to show that τ is identity on every infinite irreducible component W_{I}. Moreover, since (by Step 1) τ(W_{I}) = W_{I} and (by (5.1)) [W_{I} : W_{I}^{τ}] < ∞, it actually suffices to consider the case W_{I} = W, namely W itself is infinite and irreducible. In this case, our aim is to show that τ is identity.

First, we consider the case that W is not of type Ã_{1} and every proper W_{J} ⊂ W is infinite. Then by combining Proposition 2.13 (2) and Lemma 5.3, we have ⟨w⟩ ∩ ⟨τ(w)⟩ = 1 for some w ∈ W of infinite order whenever τ ≠ id_{S}. This implies that W^{τ} ∩ ⟨w⟩ = 1, contradicting (5.1). Thus τ must be identity now, as desired. On the other hand, the claim also holds if type W = Ã_{1}, since now we have W^{τ} = 1 whenever τ ≠ id_{S}.

Finally, we consider the remaining case that a proper W_{J} ⊂ W is infinite. We may assume that J = S ∖ {s} for some s ∈ S, so it suffices to show that τ|_{J} = id_{J}. Since |S| < ∞, we may assume further that W_{J} is irreducible: indeed, if W_{J} is not irreducible and W_{K} is an infinite irreducible component of W_{J} (which exists since |J| < ∞), then the set S ∖ {s′}, where s′ is an element of J ∖ K farthest from s in Γ, possesses the desired properties. Now Step 1 implies that τ(J) = J, so [W_{J} : W_{J}^{τ}] < ∞ (by (5.1)), therefore the induction on |S| shows that τ|_{J} = id_{J}, as desired.

Step 3: if I ∈ τ\S, then |I| < ∞.

Assume contrary that |I| = ∞. Then for any w ∈ W_{I} with J = supp(w) (finite and) nonempty, we have J ≠ I and so J ≠ τ(J) (since I is a ⟨τ⟩-orbit), therefore J ⊈ τ(J) and w ∉ W_{τ(J)}. This means that τ(w) ≠ w. Thus we have W^{τ} ∩ W_{I} = 1, contradicting (5.1).

Step 4: τ is identity on every infinite irreducible component W_{I}.

First, we consider the case that a (not necessarily proper) W_{J} ⊆ W_{I} of finite rank is infinite. We can take an irreducible W_{J}. Now assume contrary that τ is not identity on W_{I}, so τ(s) ≠ s for some s ∈ I. Then, since W_{I} is irreducible and |J| < ∞, an irreducible W_{K} ⊆ W_{I} of finite rank contains both W_{J} and s. This W_{K} is also infinite, so τ(K) = K (Step 1), therefore [W_{K} : W_{K}^{τ}] < ∞ by (5.1). Now Step 2 implies that τ is identity on W_{K}, contradicting the choice of s. Hence the claim holds in this case.

In the remaining case, W_{I} is of type A_{∞}, A_{±∞}, B_{∞} or D_{∞} (see Figure 3) as mentioned in Section 2.2. Note that τ(W_{I}) = W_{I}, since otherwise we have W^{τ} ∩ W_{I} = 1, contradicting (5.1). Now the claim is trivial in the first and the third cases where Aut Γ = {id_{S}}.

In the case type W_{I} = A_{±∞}, if τ is not identity on W_{I}, then Step 3 implies that τ is a turning of the infinite path Γ_{I}, so there is an infinite J ⊂ I with J ∩ τ(J) = ∅. Now we have W^{τ} ∩ W_{J} = 1, contradicting (5.1). This verifies the claim.

Finally, in the case type W_{I} = D_{∞}, if τ is not identity on W_{I}, then τ(s_{1}) = s_{2}, τ(s_{2}) = s_{1} and τ fixes J = I ∖ {s_{1}, s_{2}} pointwise. Put K = J ∪ {s_{2}}. Since any w ∈ W_{K} satisfies that τ(w) ∈ W_{J∪{s_{1}}}, we have W^{τ} ∩ W_{K} = W_{J} (see (2.4)), so [W_{K} : W_{J}] < ∞ by (5.1). However, putting γ_{k} = ∑_{i=2}^{k} α_{s_{i}} ∈ Φ_{K}^{+} for k ≥ 3, Lemma 2.11 implies that all of the infinitely many reflections s_{γ_{k}} belong to distinct cosets in W_{K}/W_{J}. This contradiction yields the claim.

Step 5: conclusion.

Assume that the “only if” part fails. Then by Step 4, W possesses infinitely many finite irreducible components W_{I_{1}}, W_{I_{2}}, … on which τ is not identity. Since every ⟨τ⟩-orbit is finite (Step 3), there is an infinite sequence s_{1}, s_{2}, … of distinct elements of S such that J = {s_{i} | i ≥ 1} satisfies that τ(J)∩J = ∅; take s_{1} as any element of I_{1} with τ(s_{1}) ≠ s_{1}, and if s_{1}, …, s_{k} are already chosen, then take s_{k+1} ∈ I_{i} where I_{i} does not intersect with the ⟨τ⟩-orbits of the preceding s_{j} and τ(s_{k+1}) ≠ s_{k+1}. Now we have W^{τ} ∩ W_{J} = 1 and |W_{J}| = ∞, contradicting (5.1). Hence the proof is concluded.

5.2 Proof of Theorem 5.2

We start with some preliminaries. Let τ ∈ Aut Γ and w ∈ W^{τ}, and denote the support of w as an element of (W^{τ} , S(W^{τ})) by supp^{τ} (w). The following (part of a) result of [12] shows a relation between supp(w) and supp^{τ} (w).

Proposition 5.8 (See [12, Proposition 3.3]). Let w = w_{0}(I_{1}) ⋯ w_{0}(I_{r}) (where w_{0}(I_{i}) ∈ S(W^{τ})) be a reduced expression of w ∈ W^{τ} with respect to S(W^{τ}). Then any expression of w obtained by replacing each w_{0}(I_{i}) with its reduced expression, with respect to S, is also reduced with respect to S. Hence supp(w) = ⋃_{i=1}^{r} I_{i}.

Secondly, we give a remark on the Coxeter graph of the Coxeter system (W^{τ} , S(W^{τ})), denoted here by Γ^{τ}. Let τ\Γ be the graph with vertex set τ\S, in which two orbits I, J ∈ τ\S are joined if and only if these sets are adjacent in Γ. Then the vertex set S(W^{τ}) of Γ^{τ} is regarded as a subset of the vertex set τ\S of τ\Γ via an embedding w_{0}(I) ↦ I. Now we have the following result on a relation between Γ^{τ} and τ\Γ.

Lemma 5.9. Under the embedding S(W^{τ}) ↪ τ\S of the vertex set, the underlying graph of Γ^{τ} is a full subgraph of τ\Γ.

Proof. Let I, J ∈ τ\S be two distinct orbits with both W_{I} and W_{J} finite. It is obvious that I and J are not adjacent in Γ^{τ} (i.e. w_{0}(I) and w_{0}(J) commute) if these are not adjacent in τ\S. Thus our remaining task is to show that w_{0}(I) and w_{0}(J) do not commute if I and J are adjacent in τ\S, namely some s ∈ I is adjacent to J in Γ. Now Lemma 2.7 (1) implies that w_{0}(J)·α_{s} ∈ Φ_{J∪{s}}^{+} ∖Φ_{{s}}, so w_{0}(I)w_{0}(J) · α_{s} ∈ Φ^{+}. On the other hand, we have w_{0}(I) · α_{s} ∈ Φ_{I}^{−}, so w_{0}(J)w_{0}(I) · α_{s} ∈ Φ^{−}. Thus we have w_{0}(I)w_{0}(J) ≠ w_{0}(J)w_{0}(I) as desired.

Moreover, note that τ\Γ is connected whenever Γ is. Indeed, for any I, J ∈ τ\S, a path in the connected graph Γ between any s ∈ I and any t ∈ J gives rise to a path in τ\Γ between I and J.

Proof of Theorem 5.2 (1). As is remarked above, the irreducibility of W yields the connectedness of τ\Γ, while the hypothesis implies that the embedding Γ^{τ} ↪ τ\Γ in Lemma 5.9 is now an isomorphism. Thus Γ^{τ} is connected as desired.

For the infiniteness of W^{τ}, assume the contrary. Then W^{τ} possesses the longest element w_{0}^{τ} with respect to S(W^{τ}). Now for any s ∈ S, belonging by the hypothesis to an I ∈ τ\S with |W_{I}| < ∞, the w_{0}^{τ} and w_{0}(I) admit a reduced expression with respect to S(W^{τ}) and S ending with w_{0}(I) and s, respectively, by Exchange Condition. Thus Proposition 5.8 implies that w_{0}^{τ} admits a reduced expression with respect to S ending with s. Since s ∈ S is arbitrary, this means that W is finite and w_{0}^{τ} is the longest element of W (see Section 2.2), contradicting the hypothesis that W is infinite. Hence the claim follows.

Proof of Theorem 5.2 (2). Note that the graph τ\Γ is connected. Since the hypothesis of (1) now fails, there is a path I_{0}I_{1} ⋯ I_{r} in τ\Γ, where I_{i} ∈ τ\S, such that w_{0}(I_{0}) ∈ supp^{τ} (w) and |W_{I_{r}}| = ∞. By choosing the shortest possible path, we may assume that |W_{I_{i}}| < ∞ and w_{0}(I_{i}) ∉ supp^{τ} (w) for 1 ≤ i ≤ r − 1. Now Lemma 5.9 says that I_{0}I_{1} ⋯ I_{r−1} is also a path in Γ^{τ}, so by applying Lemma 2.7 (2) to the Coxeter system (W^{τ} , S(W^{τ})), it is deduced that w_{0}(I_{r−1}) ∈ supp^{τ} (w′ww′^{−1}) where w′ = w_{0}(I_{r−1}) ⋯ w_{0}(I_{2})w_{0}(I_{1}) ∈ W^{τ}. Thus Proposition 5.8 implies that supp(w′ww′^{−1}) ⊆ S contains I_{r−1}, does not intersect I_{r} and is adjacent to I_{r} in Γ.

Take s ∈ I_{r} adjacent to supp(w′ww′^{−1}) in Γ. Now we show that, if W_{I_{r}} possesses an element u′ of infinite order such that s ∈ supp(u′^{k}) for all 0 ≠ k ∈ ℤ, then u = w′^{−1}τ^{−1}(u′)w′ is the desired element. Indeed, for k ≠ 0, we have τ^{−1}(u′)^{k}w′ww′^{−1}u′^{−k} ≠ w′ww′^{−1} by the choice of u′ and Lemma 2.7 (3) (note that τ^{−1}(u′) ∈ W_{I_{r}}), so, since τ(w′) = w′, we have

u^{k}wτ(u)^{−k} = w′^{−1}(τ^{−1}(u′)^{k}w′ww′^{−1}u′^{−k})w′ ≠ w.

Finally, we show the existence of such an element u′, concluding the proof. Since |W_{I_{r}}| = ∞ and I_{r} ∈ τ\S is a finite orbit, an irreducible component of W_{I_{r}}, therefore that containing s, is infinite. Now Theorem 4.1 implies that a Coxeter element of this component possesses the desired property.

6 Proof of the main theorem

This section is devoted to the proof of Theorem 3.1. First, note that the factor W (O_{ρ}) in the statement is ρ(G)-invariant, so the product W (O_{ρ})⟨G_{ρ}⟩ of two subgroups of W ⋊ G is indeed the semidirect product W (O_{ρ}) ⋊ ⟨G_{ρ}⟩. This implies that, since W (O_{ρ}) is generated by involutions, the claim (2) follows immediately from (1). So we prove (1) below.

For the “only if” part, we assume that wg ∈ (W ⋊ G)_{ACI} and prove that w ∈ W (O_{ρ}) and g ∈ G_{ρ} ∪ {1}. Now by (2.2) and Corollary 2.3 (2), we have

[H : Z_{H}(w′g′)] < ∞ for any H ≤ W ⋊ G and w′g′ ∈ ⟨wg⟩_{◁W ⋊G}.  (6.1)

Note that ρ_{g}(w) = w^{−1} and g^{2} = 1 since 1 = (wg)^{2} = wρ_{g}(w) · g^{2}. We divide the proof into the following five steps.

Step 1: ρ_{g} maps each W_{I} ∈ C_{W}^{inf} onto itself.

Assume contrary that ρ_{g} maps W_{I} onto an irreducible component other than W_{I}. Let π : W ↠ W_{I} be the projection. Take s ∈ I and put a = swgs(wg)^{−1} ∈ ⟨wg⟩_{◁W ⋊G}. Then we have a = swρ_{g}(s)w^{−1} ∈ W, so Z_{W_{I}} (a) = Z_{W_{I}} (π(a)). Thus (6.1) implies that π(a) ∈ W_{I} is almost central in W_{I}. However, the first assumption yields that π(ρ_{g}(s)) = 1, so π(a) = sπ(w)1π(w)^{−1} = s, which is not almost central in W_{I} by Proposition 2.15. This is a contradiction.

Step 2: ρ_{g} is identity on every W_{I} ∈ C_{W}^{inf}.

Assume that the claim fails for W_{I}. Note that ρ_{g}(W_{I}) = W_{I} by Step 1. Let π : W ↠ W_{I} be the projection. Then we may assume without loss of generality that ℓ(π(w)) ≤ ℓ(π(uwρ_{g}(u)^{−1})) for all u ∈ W_{I}; if this inequality fails, replace wg with another involution u(wg)u^{−1} = uwρ_{g}(u)^{−1} · g in ⟨wg⟩_{◁W ⋊G}, which is also almost central in W ⋊ G by (6.1), and use the induction on ℓ(π(w)).

Put τ = ρ_{g}|_{I} ∈ Aut Γ_{I}, which is assumed to be non-identity. Now if π(w) = 1, then we have Z_{W_{I}} (wg) = Z_{W_{I}} (g) = W_{I}^{τ} and so [W_{I} : W_{I}^{τ}] < ∞ by (6.1), contradicting Theorem 5.1. Thus π(w) ≠ 1.

We show that π(w) is an involution in W_{I}^{τ}. Let s_{1} ⋯ s_{n} (where n ≥ 1 and s_{i} ∈ I) be an arbitrary reduced expression of π(w) ∈ W_{I}. Then, since ρ_{g}(w) = w^{−1} and ρ_{g}(W_{I}) = W_{I}, we have

π(w) = π(ρ_{g}(w)^{−1}) = ρ_{g}(π(w)^{−1}) = τ(π(w)^{−1}) = τ(s_{n}) ⋯ τ(s_{1}),

so ℓ(π(w)τ(s_{1})) < ℓ(π(w)), therefore Exchange Condition shows that π(w) = s_{1} ⋯ ŝ_{i} ⋯ s_{n}τ(s_{1}) for an index i. Now if i ≥ 2, then π(s_{1}wτ(s_{1})^{−1}) = s_{2} ⋯ ŝ_{i} ⋯ s_{n} is shorter than π(w), contradicting the minimality of ℓ(π(w)). Thus we have i = 1 and π(w) = s_{2} ⋯ s_{n}τ(s_{1}). Since the original reduced expression s_{1} ⋯ s_{n} is arbitrary, we can apply this argument to the new expression of π(w). Iterating, we have

π(w) = s_{3} ⋯ s_{n}τ(s_{1})τ(s_{2}) = ⋯ = s_{n}τ(s_{1}) ⋯ τ(s_{n−1}) = τ(s_{1}) ⋯ τ(s_{n}).

Since π(w) = s_{1} ⋯ s_{n} = τ(s_{n}) ⋯ τ(s_{1}), the claim of this paragraph follows.

Now if m_{s,τ(s)} = ∞ for some s ∈ I, then since τ^{2} = id_{I}, Theorem 5.2 (2) (applied to π(w)) gives us an element u ∈ W_{I} of infinite order such that

π(u^{k}wgu^{−k}(wg)^{−1}) = u^{k}π(w)τ(u)^{−k}π(w)^{−1} ≠ 1 for all k ≠ 0.

This means that Z_{⟨u⟩}(wg) = 1, so [⟨u⟩ : Z_{⟨u⟩}(wg)] = ∞, contradicting (6.1). On the other hand, if m_{s,τ(s)} < ∞ for all s ∈ I, then the Coxeter group W_{I}^{τ} is infinite and irreducible by Theorem 5.2 (1). Now we have

Z_{W_{I}}^{τ} (π(w)) = Z_{W_{I}}^{τ} (w) = Z_{W_{I}}^{τ} (wg),

which has finite index in W_{I}^{τ} by (6.1). Thus the non-identity involution π(w) ∈ W_{I}^{τ} is almost central in W_{I}^{τ}, contradicting Proposition 2.15. Hence Step 2 is concluded.

Step 3: w ∈ W_{fin}.

We show that π(w) = 1 for any W_{I} ∈ C_{W}^{inf} with projection π : W ↠ W_{I}. Since ρ_{g} is identity on W_{I} (Step 2), we have Z_{W_{I}} (wg) = Z_{W_{I}} (w) = Z_{W_{I}} (π(w)) and so (by (6.1)) π(w) is almost central in W_{I}. Now since 1 = π(wρ_{g}(w)) = π(w)ρ_{g}(π(w)) = π(w)^{2}, the claim follows from Proposition 2.15.

Step 4: w ∈ W (O_{ρ}).

Assume the contrary. Then there exist a ρ^{†}(G)-orbit O ⊆ C_{W}^{fin} with infinite cardinality and W_{I} ∈ O (with projection π_{I} : W ↠ W_{I}) such that π_{I}(w) ≠ 1. Fix the O, and let O_{0} be the set of all such W_{I} ∈ O, so |O_{0}| < ∞.

We show that ρ_{h}^{†}(O_{0}) = O_{0}, or equivalently ρ_{h}^{†}(O_{0}) ⊆ ρ_{h}^{†}(O_{0}), for any h ∈ Z_{G}(wg). Note that ρ_{h}(w) = w since wgh = hwg = ρ_{h}(w)hg. Now if W_{I} ∈ O_{0} and ρ_{h}^{†}(W_{I}) = W_{J} ∉ O_{0}, then π_{J}(ρ_{h}(w)) = ρ_{h}(π_{I}(w)) ≠ 1 and π_{J}(w) = 1, contradicting ρ_{h}(w) = w. Thus ρ_{h}^{†}(O_{0}) ⊆ O_{0} as desired.

Since O is an infinite ρ^{†}(G)-orbit, we can choose infinitely many finite subsets O_{1}, O_{2}, … of O, irreducible components W_{I_{0}}, W_{I_{1}}, W_{I_{2}}, ⋯ ∈ O and elements g_{0}, g_{1}, g_{2}, ⋯ ∈ G inductively, where we start with arbitrary W_{I_{0}} ∈ O_{0} and g_{0} = 1, subject to the conditions W_{I_{k}} ∉ ⋃_{i=0}^{k−1} O_{i}, ρ_{g_{k}}^{†}(W_{I_{0}}) = W_{I_{k}} and O_{k} = ρ_{g_{k}}^{†}(O_{0}) ∋ W_{I_{k}} for all k ≥ 1. Now if i < j and h ∈ Z_{G}(wg), then the previous paragraph implies that ρ_{g_{i}h}^{†}(W_{I_{0}}) = ρ_{g_{i}}^{†}ρ_{h}^{†}(W_{I_{0}}) ∈ ρ_{g_{i}}^{†}(O_{0}) = O_{i}, while ρ_{g_{j}}^{†}(W_{I_{0}}) = W_{I_{j}} ∉ O_{i}, so we have g_{j} ≠ g_{i}h. Thus all the g_{i} belong to distinct cosets in G/Z_{G}(wg), while [G : Z_{G}(wg)] < ∞ by (6.1). This contradiction yields the claim.

Step 5: g ∈ G_{ρ} ∪ {1}.

Note that g^{2} = 1. Since hwg = ρ_{h}(w) · hg for h ∈ G, we have Z_{G}(wg) ⊆ Z_{G}(g) and so g is almost central in G by (6.1). From now, we check (3.1).

By Step 4, the union O of a finite number of some ρ^{†}(G)-orbits with finite cardinalities satisfies that w ∈ W (O). Since |O| < ∞, it suffices to show that ρ_{g} is identity on all W_{I} ∈ O′ = C_{W} ∖ O except a finite number of finite irreducible components. Now O′ is ρ_{g}^{†}-invariant as well as its complement O, while W (O′) ⊆ Z_{W} (w), so Z_{W (O′)}(wg) = Z_{W (O′)}(g) is the fixed-point subgroup W (O′)^{τ} (where τ = ρ_{g}|_{W (O′)}). Since Z_{W (O′)}(wg) has finite index in W (O′) by (6.1), the claim follows from Theorem 5.1.

Hence the “only if” part has been proved. From now, we prove the other part; so we assume that w ∈ W (O_{ρ}), g ∈ G_{ρ} ∪ {1} and wg is an involution, and prove that wg is almost central in W ⋊ G. By the choice of w, there are a finite number of finite ρ^{†}(G)-orbits in C_{W}^{fin} such that their union O satisfies that w ∈ W (O). Now note that

Z_{W ⋊G}(wg) ⊇ Z_{W ⋊G}(w) ∩ Z_{W ⋊G}(g) ⊇ (Z_{W} (w)Z_{G}(w)) ∩ (Z_{W} (g)Z_{G}(g)),

so it suffices to show that both Z_{W} (w)Z_{G}(w) and Z_{W} (g)Z_{G}(g) have finite index in W ⋊ G (see (2.3)). Moreover, Lemma 2.4 reduces the claim to the following four claims:

Step 6: Z_{W} (w) has finite index in W.

This follows since w lies in the finite direct factor W (O) of W.

Step 7: Z_{G}(w) has finite index in G.

Since W (O) is finite and ρ(G)-invariant, the action gives rise to a homomorphism ρ′ from G to the finite group Aut W (O). Now ker ρ′ is contained in Z_{G}(w) (since w ∈ W (O)) and has finite index in G, proving the claim.

Step 8: Z_{W} (g) has finite index in W.

This is trivial if g = 1. If g ∈ G_{ρ}, then the property (3.1) and Theorem 5.1 imply that the fixed-point subgroup W^{ρ_{g}} = Z_{W} (g) by ρ_{g} has finite index in W, as desired.

Step 9: Z_{G}(g) has finite index in G.

This is obvious from the choice of g.

Hence the proof of Theorem 3.1 is concluded.

Bibliography

[1] P. Bahls, The Isomorphism Problem in Coxeter Groups, Imperial College Press, London, 2005.

[2] N. Brady, J. P. McCammond, B. Mühlherr, W. D. Neumann, Rigidity of Coxeter groups and Artin groups, Geom. Dedicata 94 (2002) 91–109.

[3] B. Brink, R. B. Howlett, Normalizers of parabolic subgroups in Coxeter groups, Invent. Math. 136 (1999) 323–351.

[4] V. V. Deodhar, On the root system of a Coxeter group, Comm. Algebra 10 (1982) 611–630.

[5] V. V. Deodhar, A note on subgroups generated by reflections in Coxeter groups, Arch. Math. (Basel) 53 (1989) 543–546.

[6] M. Dyer, Reflection subgroups of Coxeter systems, J. Algebra 135 (1990) 57–73.

[7] W. N. Franzsen, R. B. Howlett, Automorphisms of Coxeter groups of rank three, Proc. Amer. Math. Soc. 129 (2001) 2607–2616.

[8] W. N. Franzsen, R. B. Howlett, Automorphisms of nearly finite Coxeter groups, Adv. Geom. 3 (2003) 301–338.

[9] J. E. Humphreys, Reflection Groups and Coxeter Groups, Cambridge Univ. Press, Cambridge, 1990.

[10] D. Krammer, The Conjugacy Problem for Coxeter Groups, Ph.D. Thesis, Univ. Utrecht, 1994.

[11] B. Mühlherr, Coxeter groups in Coxeter groups, in: Finite Geometry and Combinatorics, London Math. Soc. Lecture Note Ser. 191, Cambridge Univ. Press, Cambridge, 1993, pp. 277–287.

[12] M. Nanba, Bruhat order on the fixed-point subgroup by a Coxeter graph automorphism, J. Algebra 285 (2005) 470–480.

[13] K. Nuida, Centralizers of reflections and reflection-independence of Coxeter groups, preprint.

[14] K. Nuida, On centralizers of parabolic subgroups in Coxeter groups, arXiv:math.GR/0501061.

[15] K. Nuida, On the direct indecomposability of infinite irreducible Coxeter groups and the Isomorphism Problem of Coxeter groups, to appear in Communications in Algebra, arXiv:math.GR/0501276.

[16] L. Paris, Irreducible Coxeter groups, arXiv:math.GR/0412214.

[17] R. W. Richardson, Conjugacy classes of involutions in Coxeter groups, Bull. Austral. Math. Soc. 26 (1982) 1–15.

[18] R. Steinberg, Endomorphisms of Linear Algebraic Groups, Mem. Amer. Math. Soc., vol. 80, 1968.

Koji Nuida Graduate School of Mathematical Sciences, University of Tokyo

3-8-1 Komaba, Meguro-ku, Tokyo, 153-8914, Japan E-mail: nuida@ms.u-tokyo.ac.jp