One Hat Cyber Team
Your IP :
216.73.216.14
Server IP :
194.44.31.54
Server :
Linux zen.imath.kiev.ua 4.18.0-553.77.1.el8_10.x86_64 #1 SMP Fri Oct 3 14:30:23 UTC 2025 x86_64
Server Software :
Apache/2.4.37 (Rocky Linux) OpenSSL/1.1.1k
PHP Version :
5.6.40
Buat File
|
Buat Folder
Eksekusi
Dir :
~
/
home
/
vo
/
book-newprint
/
View File Name :
chapt23.tex
\section[Representations of two-dimensional Lie algebras, their nonlinear transformations, and semilinear relations]{Representations of two-dimensional Lie algebras, their nonlinear transformations, and\break semilinear rela\-ti\-ons} \label{sec:1.3} \markright{1.3. Lie algebras and semilinear relations} \subsection{Representations of two-dimensional real Lie algebras and their nonlinear transformations by bounded operators}\label{sec:1.3.1} \textbf{1.} Lie algebras $(IV_0)$, $(IV_1)$, $(IV_2)$, similarly to the relations $(V_0)$, $(V_1)$, $(V_1')$, can be treated from a general point of view. Namely, all these relations are partial cases of the relation \begin{equation}\label{deformedlie} [A,B] = iP_2(A), \end{equation} where $P_2(A)$ is a real quadratic polynomial. If $A= A^*$, $B=B^* \in L(H)$, then their study can be performed by using the following theorem. \begin{theorem} \textup{(D. Kleinecke, F.V. Shirokov).} If $P$ and $Q$ are bounded operators, and $[P,[P,Q]]=0$, then the operator $[P, Q]$ is quasi-nilpotent, i.e., \[ \lim_{n \to \infty} \sqrt[n]{\|[P, Q]^n\|} =0. \] \end{theorem} \begin{proof} The Kleinecke--Shirokov theorem follows, for example, from the formula $\ad_P^nQ^n= n!\, (\ad_PQ)^n$, where $\ad_PX = PX - XP$ is a bounded operator in $L(H)$, and $\|\ad _P\| \le 2 \|P\|$. Then \par\vskip\abovedisplayskip\noindent \hfill $\displaystyle\sqrt[n]{\|[P,Q]^n\|} \le \frac 2{\sqrt[n]{n!}} \, \|P\|\cdot \|Q\| \to 0, \qquad n \to \infty.$ \hfill \end{proof} In Section \ref{sec:1.3.3} below we will give many analogies of this theorem. \medskip\noindent\textbf{2.} Now we consider bounded self-adjoint operators satisfying~\eqref{deformedlie}. \begin{proposition} Irreducible pairs, $A$, $B$ of bounded self-adjoint operators which satisfy the relation~\eqref{deformedlie} are one-dimensional, and they are given by\textup: $A=\lambda$, $B=\mu$, where $(\lambda, \mu) \in M = \{(\lambda, \mu) \in \mathbb{R}^2 \mid P_2(\lambda) =0\}$. An arbitrary bounded pair has the form \[ A = \int_M \lambda \, dE(\lambda, \mu), \quad B = \int_M \mu\, dE(\lambda, \mu), \] where $E(\cdot, \cdot)$ is the resolution of the identity on $M$. \end{proposition} \begin{proof} Indeed, \eqref{deformedlie} implies $[A, [A,B]]=0$, and by the Kleinecke--Shirokov theorem, the operator $[A,B]$ is quasi-nilpotent. But since $[A,B]$ is skew-adjoint, it yields $[A,B]=0$. Then $P_2(A)=0$, and the statement follows from the spectral theorem for a pair of commuting self-adjoint operators. \end{proof} \begin{remark} The proposition above implies that if the polynomial $P_2(\cdot)$ has no real roots, then there are no bounded self-adjoint pairs that satisfy \eqref{deformedlie}. In particular, there are no bounded pairs that satisfy CCR (relation $(IV_1)$), or $[A,B] = i(A^2 +I)$ (relation $(V_1)$). \end{remark} \begin{remark} Relation $(V_2)$ $[A,B] = i(A^2 +B)$ can also be rewritten in the equivalent form $[A, A^2 +B] = i(A^2 +B)$, which can be reduced by a non-degenerate non-linear change of variables, $\tilde A = A^2 +B$, $\tilde B = B$, to the form $[\tilde A, \tilde B] =i\tilde A$. Since $\tilde A$ and $\tilde B$ are also bounded self-adjoint operators, by the Kleinecke--Shirokov theorem, we have $[\tilde A, \tilde B]=0$, and $[A,B]=0$, which yields $A^2 =-B$. Irreducible representations of relation$ (V_2)$ are one-dimensional, $A=\lambda$, $B=\mu$, $(\lambda, \mu) \in M_{(V_2)} = \{ (\lambda, \mu) \in \mathbb{R}^2 \mid \lambda^2 = -\mu\}$. An arbitrary pair of bounded self-adjoint operators satisfying relation $(V_2)$ has the form \[ A = \int_{M_{(V_2)}} \lambda\, dE(\lambda, \mu), \quad A = \int_{M_{(V_2)}} \lambda\, dE(\lambda, \mu), \] where $E(\cdot, \cdot)$ is the resolution of the identity on $M_{(V_2)}$. \end{remark} \noindent\textbf{3.} If in the study of representations of relations given by the families $(IV)$ and $(V)$, we restrict ourselves to only the finite-dimensional level, then instead of the Kleinecke--Shirokov theorem, which was proved in 1956, we could apply its finite-dimensional variant, which is the following theorem proved much earlier (1935). We formulate it in the simplest case. \begin{theorem} \textup{(N. Jacobson).} If $\dim H < \infty$ and $[P,[P,Q]]=0$, then the operator $[P,Q]$ is nilpotent. \end{theorem} \begin{remark} In the relations $(IV)$ and $(V)$ no new irreducible representations arise when passing from the finite-dimensional to the bounded case. Of course, considering unbounded operators, for which the Klei\-necke--Shirokov theorem fails, one obtains a much more consistent representation theory of the relations $(IV)$ and $(V)$. \end{remark} \subsection{Pairs of operators connected by semilinear relations} \label{sec:1.3.2} In Sections \ref{sec:1.3.2}--\ref{sec:1.3.5} we study a wide class of so-called semilinear relations which connect pairs of bounded operators $A=A^*$, $B\in L(H)$. They appear as a generalization of the relations $(IV)$ and $(V)$ studied in previous sections. We prove many Kleinecke--Shirokov type theorems (Section~\ref{sec:1.3.3}). For general semilinear relations, we describe properties of irreducible representations (Section~\ref{sec:1.3.4}). All irreducible representations are classified up to unitary equivalence for semilinear ${ F}_4$-relations (Section~\ref{sec:1.3.5}). \medskip\noindent\textbf{1.} Consider bounded operators $A=A^*$, $B\in L(H)$, which satisfy a relation of the form: \begin{equation}\label{sr} \sum_{i=1}^nf_i(A)Bg_i(A)=h(A), \end{equation} where $f_i(\cdot)$, $g_i(\cdot)$, $h(\cdot)$, $i=1$, \dots, $n$, are complex bounded Borel mappings defined on ${\mathbb R}$ or a subset $D$, $\sigma(A) \subset D$. Relation (\ref{sr}) is called semilinear and the operators $A$, $B$ are called representation of (\ref{sr}). If $f_i(\cdot)$, $g_i(\cdot)$, $h(\cdot)$, $i=1$, \dots, $ n$, are polynomials, then the operators $A$, $B$, $B^*$ define a representation of the $*$-algebra $\mathfrak A$ with three generators $a=a^*$, $b$, $b^*$ satisfying the relation $\sum_{i=1}^nf_i(a)\,b\,g_i(a)=h(a)$. This algebra is the quotient algebra of the free $*$-algebra with three generators ${\mathbb C}\langle a=a^*, b,b^*\rangle$ with respect to the two-sided $*$-ideal generated by the element $\sum_{i=1}^nf_i(a)\,b\,g_i(a)-h(a)$. Unless otherwise stated, we assume here $f_i$, $g_i$, $h$ to be defined on the whole ${\mathbb R}$. The general case can be easily derived from this one. \begin{remark} \rm As before, if the operators $A=A^*$, $B$ are unbounded, then it is necessary to define the meaning of the operator equality (\ref{sr}). We investigate the question of the ``correct'' definition of relation (\ref{sr}) with unbounded operators for some special relations in \cite{sam_tur_sh}. \end{remark} \noindent\textbf{2.} The study of bounded representations of (\ref{sr}) can be reduced to the study of operators satisfying the corresponding homogeneous relation: \begin{equation}\label{hom} \sum_{i=1}^nf_i(A)Bg_i(A)=0. \end{equation} If the function $\phi (t)=h(t)/\sum_{i=1}^nf_i(t)g_i(t)$ is bounded on the spectrum of the operator $A=A^*$, then the pair of operators $A$, $B=\phi (A)$ is a particular solution of the inhomogeneous relation (\ref{sr}), and the general solution of (\ref{sr}), with the operator $A$ fixed, consists of all pairs ($A$, $B+\phi(A)$) where ($A$, $B$) satisfies (\ref{hom}). Let $$ \tilde\phi(t)= \begin{cases} h(t)/\sum_{i=1}^nf_i(t)\,g_i(t), &\mbox{if}\ \sum_{i=1}^nf_i(t)\,g_i(t)\ne 0,\\ 0,&\mbox{otherwise}. \end{cases} $$ \begin{proposition} If \eqref{sr} has a \textup(bounded\textup) solution $(A,B)$, then there exists $C>0$ such that $$ |h(t)|\le C\Bigl|\sum_{i=1}^nf_i(t)g_i(t)\Bigr|, \qquad t\in \sigma(A). $$ Any solution $A$, $B$ of \eqref{sr} has the form $B=B'+\tilde\phi(A)$ where $(A, B')$ is a representation of \eqref{hom}. \end{proposition} \begin{proof} Let $A$, $B$ satisfy \eqref{sr}, and let $\psi\colon L(H)\rightarrow {\mathfrak A}$ be a conditional expectation on the $W^*$-algebra $\mathfrak A$ of all operators which commute with the operator $A$. Applying $\psi$ to both sides of equation \eqref{sr}, we obtain $$ \sum_{i=1}^nf_i(A)\,\psi (B)\,g_i(A)=h(A). $$ From this we conclude that $(A, \psi(B))$ is a solution of \eqref{sr} and $$ \psi (B)\sum_{i=1}^nf_i(A)\,g_i(A)=h(A). $$ Thus, $\psi(B)=\tilde\phi(A)$ on the image of the operator $\sum_{i=1}^n f_i(A)\,g_i(A)$. Since $\psi(B)$ is a bounded operator, there exists $C>0$ such that for every $t\in\sigma(A)$ and $\sum_{i=1}^nf_i(t)\,g_i(t)\ne 0$ we have \[ |h(t)|\le C\Bigl|\sum_{i=1}^nf_i(t)g_i(t)\Bigr|. \] On the other hand, $\ker \sum_{i=1}^nf_i(A)\,g_i(A)\subset \ker h(A)$, which implies that the inequality holds for every $t\in\sigma(A)$, and $(A, B-\tilde\phi(A))$ is a representation of relation (\ref{hom}). \end{proof} It is easy to prove that the correspondence between irreducible representations of (\ref{sr}) and (\ref{hom}) preserves unitary equivalence. So, in studying bounded representations, we can restrict ourselves only to the homogeneous relations (\ref{hom}). \medskip\noindent\textbf{3.} To the semilinear relation \eqref{hom}, we associate: \begin{itemize} \item[a)] the characteristic function: $$ \Phi(t,s)=\sum_{i=1}^nf_i(t)\,g_i(s),\qquad (t,s\in {\mathbb R}); $$ \item[b)] the characteristic binary relation: $$ \Gamma=\{(t,s)\mid \Phi(t,s)=0\}\subset {\mathbb R}^2; $$ \item[c)] an oriented graph $({\mathbb R}, \Gamma)$, where the edge \vrule width0pt height 0pt depth 8pt$\edgeright{t}{s}$, $t$, $s\in {\mathbb R}$, belongs to the graph if and only if $\Phi(t,s)=0$. \end{itemize} If no confusion arise, we will write simply $\Gamma$ for the graph ($D$, $\Gamma$) and call it the graph of relation (\ref{hom}). Any subset $M\subset {\mathbb R}$ determines a subgraph $\Gamma\restriction_M$ such that its vertices are points of $M$ and its edges are edges of $\Gamma$ which connect points of $M$. If $\Phi(t,s)=0$ is equivalent to $\Phi(s,t)=0$, the graph $\Gamma$ together with the edge $\edgeright{}{}$ also contains the edge $\edgeleft{}{}\!\!$. In this case, we will consider the graph as non-oriented. In what follows $\Gamma_s$ denote the set $\{(t,s)\mid \Phi(t,s)=0,\, \Phi(s,t)=0\}\subset {\mathbb R}^2$ or the corresponding non-oriented graph. Consider some examples of semilinear relations. 1) Relations $\ad_A(B)=[A,B]=AB-BA=0$ and $(\ad_A)^n(B)=[A,\dots[A,B]\dots]=0$ have the characteristic functions of the form\allowbreak{} $\Phi(t,s)=t-s$ and $\Phi(t,s)=(t-s)^n$ respectively, and define the same graph, all connected components of which are the following: \vrule width0pt height0pt depth8pt$\ear {\lambda}$, $\lambda\in{\mathbb R}$. 2) Characteristic functions corresponding to the relations \[ \ad_{A,-1}(B)=\{A,B\}=AB+BA=0 \] and $(\ad_{A,-1})^n(B)=\{A,\ldots\{A,B\}\ldots\}=0$ are $\Phi(t,s)=t+s$ and $\Phi(t,s)=(t+s)^n$, respectively. As before, they define the same graph with connected components of the form: $$ \ear {0},\quad \edge {\lambda}{\smash{\hbox{\kern-5pt$\scriptstyle-\lambda$}}},\qquad ({\lambda>0}). $$ 3) Let $ABA=\alpha B$, $\alpha\in{\mathbb R}$. Then $\Phi(t,s)=ts-\alpha$, and all connected components of the corresponding graph are of the form: a) $\alpha\ne 0$ $$ \edge {\alpha\lambda^{-1}} {\lambda},\qquad({ \lambda^2\ne\alpha,0})\qquad \onepoint {0},\qquad \ear{\smash{\pm\sqrt{\alpha}}}\ \ ,\qquad ({\alpha>0}); $$ b) if $\alpha=0$, then any vertex $\lambda\in{\mathbb R}\setminus\{0\}$ is connected with $0$ by the edge \vrule width0pt height0pt depth8pt$\edge{\lambda}{0}$. 4) Any connected component of the graph corresponding to the relation $$ A^2B-(q+q^{-1})ABA+BA^2=0,\qquad q\in{\mathbb R}\setminus\{-1,0,1\}, $$ is either an infinite chain: $$ \qquad\chainl{\smash{q^{-1}t}}{t}{\smash{qt}} ,\qquad t\ne 0, $$ or \vrule width0pt height0pt depth8pt$\ear{0}.$ \subsection{Kleinecke--Shirokov type theorems} \label{sec:1.3.3} We begin with the study of the structure of operators $A=A^*$, $B$ satisfying \eqref{hom}; in particular, we will look at the connection between the spectral properties of the operator $A$ and the structure of the operator $B$. It turns out that for a broad class of semilinear relations the characteristic binary relation corresponding to them completely defines solutions of \eqref{hom}. This fact provides many Kleinecke--Shirokov type theorems. \medskip\noindent\textbf{1.} We start by studying finite-dimensional representations of semilinear relations. Note that the same arguments work in the more general case where the operator $A$ in the representation has discrete spectrum. So let $\sigma (A)=\{\lambda_1,\ldots, \lambda_m\}$ and $H_{\lambda_j}$ be eigenspaces of $A$ corresponding to $\lambda_j$. With respect to the decomposition $H=H_{\lambda_1}\oplus \ldots \oplus H_{\lambda_m}$, the operator $B$ can be written in the form of the block matrix: $ B=(B_{ln})_{l,n=1}^m. $ \begin{proposition}\label{pr1} For operators $A$, $B$ to define a representation of the relation \eqref{hom}, it is necessary and sufficient that the block matrix $B=(B_{sn})_{s,n=1}^m$ be supported by $\Gamma\restriction_{\sigma (A)}$ \textup(i.e., $B_{sn}=0$ for any $(\lambda_s, \lambda_n)\notin \Gamma$\textup). \end{proposition} \begin{proof} The statement follows immediately from the equality \par\vskip\abovedisplayskip\hfill$\displaystyle \Bigl(\sum _{i=1}f_i(A)\,B\,g_i(A)\Bigr)_{kj}= \Phi (\lambda_k,\lambda_j)\,B_{kj},\qquad k,j=1,\ldots,m. $\hfill \end{proof} \noindent\textbf{2.} A pair $A=A^*$, $B=B^*$, is a representation of relation \eqref{hom} if and only if the block matrix $B=(B_{ij})_{i,j=1}^n$ is supported by $$ \Gamma_s|_{\sigma(A)}=\{(t,s)\in\sigma(A)\times\sigma(A)\mid \Phi(t,s)=\Phi(s,t)=0\}. $$ In fact, if $A=A^*$, $B=B^*$ is a representation of (\ref{hom}), then $A$, $B$ also satisfy the relation $$ \sum_{i=1}^n\bar g_i(A)\,B\,\bar f_i(A)=0. $$ Hence the block matrix $B=(B_{ij})_{i,j=1}^m$ is supported by $(\Gamma\cap\Gamma^*)|_{\sigma(A)}$, where $\Gamma^*=\{(t,s)\in {\mathbb R}\times{\mathbb R} \mid \Phi(s,t) =\sum_{i=1}^ng_i(t)\,f_i(s)=0\}$, which gives the required statement. \medskip\noindent\textbf{3.} Now we will try to remove the condition in Proposition~\ref{pr1} that $\sigma(A)$ is discrete. For this purpose, we will consider a more general situation. Let $M$, $N$ be normal operators acting on Hilbert spaces $H_M$, $H_N$, respectively, and let $E_{M}(\cdot)$, $E_N(\cdot)$ be their spectral measures. \begin{definition}\label{def1} We say that a subset ${ F}\subset{\mathbb C} \times{\mathbb C} $ $(M,N)$-supports an operator $B\colon H_N\rightarrow H_M$ if $$ E_M(\alpha)\,B\,E_N(\beta)=0 $$ for any pair $(\alpha,\beta)$ of Borel sets such that $(\alpha\times\beta)\cap{ F}=\emptyset$. \end{definition} It is not difficult to prove that there exists a smallest closed set ${ F}$ supporting $B$ (take the complement to the union of all such open sets $\alpha\times\beta$). We will denote this set by $\supp_{M,N}(B)$. It is clear that $\supp_{M,N}(B)\subset\sigma(M)\times\sigma(N)$; in particular, it is a subset of ${\mathbb R} \times{\mathbb R} $ when $M$ and $N$ are self-adjoint. We shall also write $\supp_M(B)$ instead of $\supp_{M,M}(B)$. \begin{theorem} \label{th1-semi} If $A$, $B$ is a representation of relation \eqref{hom}, then $$ \supp_A(B)\subset \Gamma, $$ where $\Gamma$ is the binary relation corresponding to \eqref{hom}. \end{theorem} \begin{proof} Let $\alpha\times \beta$ do not intersect $\Gamma$. Denote by $\mu$ the scalar spectral measure of the operator $A$. By Luzin's theorem, for every $\varepsilon>0$, there exist compact sets $\alpha'\subset \alpha$ and $\beta'\subset \beta$ such that $\mu(\alpha\setminus \alpha')<\varepsilon$, $\mu(\beta\setminus \beta')<\varepsilon$, and the functions $f_i'=f_i\restriction_{\alpha'}$, $g_i'=g_irestriction_{\beta'}$ are continuous. Put $P=E_{A}(\alpha')$, $Q=E_{A}(\beta')$, $B'=PBQ\colon QH\rightarrow PH$. The operator of multiplication, $$ \Delta\colon X\mapsto \sum_{i=1}^nf_i'(A)\,X\,g_i'(A), $$ acts on the space $L(QH, PH)$. The spectrum of the operator $\Delta$ belongs to the set $$ \Bigl\{\sum_{i=1}^nf_i(\lambda)\,g_i(\mu)\mid\lambda \in \alpha',\,\mu \in \beta'\Bigr\}= \Phi(\alpha'\times \beta'), $$ see, for example \cite{137}. By the condition of the theorem, $\Phi(\alpha'\times \beta')$ does not contain zero, hence the operator $\Delta$ is invertible. Since $\Delta(B')=0$, we have that $B'=0$, i.e., $E_A(\alpha')\,BE_A(\beta')=0$. Letting $\varepsilon \rightarrow 0$, we obtain that $E_A(\alpha)\,BE_A(\beta)=0$. \end{proof} In a similar way, more general results can be proved. \begin{theorem} \label{th2-semi} If $M$, $N$ are normal operators and \begin{equation}\label{p20'} \sum_{i=1}^nf_i(M)\,B\,g_i(N)=0, \end{equation} then $$ \supp_{M,N}(B)\subset \Gamma, $$ where $\Gamma=\bigl\{(t,s)\in {\mathbb C} \times{\mathbb C} \mid \Phi(t,s)=\sum_{i=1}^n f_i(t)\,g_i(s)=0\bigr\}$. \end{theorem} \medskip\noindent\textbf{4.} For any $F\subset {\mathbb C} \times{\mathbb C} $ let us denote by ${\mathfrak M}(F)$ the set of all operators supported by $F$. If necessary, we will write more explicitly: ${\mathfrak M}_A(F)$ or ${\mathfrak M}_{M,N}(F)$. Similarly, we will denote by $\Delta$ (instead of $\Delta_{M,N}$ or $\Delta_A$) the multiplication operator $$ X\mapsto\sum_{i=1}^nf_i(M)\,X\,g_i(N) $$ on $L(H_N,H_M)$. As analogue of Proposition~\ref{pr1}, it would be natural to expect that the equality \begin{equation}\label{p20} {\mathfrak M}(\Gamma)=\ker\Delta \end{equation} holds. Theorem~\ref{th1-semi} shows that ${\mathfrak M}(\Gamma)\supset\ker\Delta$. We shall see that the inverse inclusion is true under some restrictions on the smoothness of the functions $f_i$, $g_i$, and it is not true in the general case. Let us establish some additional results. Denote by $\pr_1$, $\pr_2$ the projections onto the components in $\sigma(M)\times\sigma(N)$. Let $S=(\sigma(M)\times\sigma(N))\cap\Gamma$. \begin{lemma}\label{l1} Let $g_i\in\Lip\sigma(N)$, $k=1$, \dots, $n$. If $\pr_1S=\sigma(M)$, then $$ \|\Delta\|\le 2\sum_{i=1}^n\|f_i\|\, \|g_i\|_{\Lip}\diam\sigma(N) $$ where $\|f_i\|=\sup\{f_i(t)\mid t\in\sigma(M)\}$. \end{lemma} \begin{proof} Set $C=2\sum_{i=1}^n\|f_i\|\, \|g_i\|_{\Lip}\diam\sigma(N)$ and fix $s_0\in \sigma(N)$. Then $$ \Delta(B)=\sum_{i=1}^nf_i(M)\,B\,g_i(s_0)+\sum_{i=1}^nf_i(M)\,B\,(g_i(N)-g_i( s_0)I). $$ Since $\sigma(g_i(N)-g_i(s_0)I)=\{g_i(t)-g_i(s_0)\mid t\in \sigma(N)\}$, we have that $$ \|g_i(N)-g_i(s_0)I\|\le \|g_i\|_{\Lip}\diam\sigma(N). $$ Thus, $$ \Bigl\|\sum_{i=1}^nf_i(M)\,B\,(g_i(N)-g_i(s_0)I)\Bigr\|\le \frac{1}{2}\,C\|B\|. $$ Furthermore, $\sigma\bigl(\sum_{i=1}^ng_i(s_0)\,f_i(M)\bigr)=\{\Phi(t,s_0)\mid t\in \sigma(M)\}$. By the condition of the theorem, for any $t\in\sigma(M)$ there exists $ s=s(t)\in \sigma(N) $ such that $\Phi(t,s)=0$. Therefore, \begin{gather*} |\Phi(t,s_0)|=|\Phi(t,s_0)-\Phi(t,s(t))| \\ \Bigl|\sum_{i=1}^n(g_i(s_0)-g_i(s(t)))\,f_i(s)\Bigr|\le \frac{1}{2}\,C. \end{gather*} Hence, $\bigl\|\sum_{i=1}^ng_i(s_0)\,f_i(M)\bigr\|\le C/2$. Thus, $$ \Bigl\|\sum_{i=1}^nf_i(M)\,B\,g_i(s_0)\Bigr\|\le \frac{1}{2}\,C\|B\|, $$ and $\|\Delta (B)\|\le C\|B\|$. \end{proof} \begin{lemma}\label{l2} Let $E(\cdot)$ be a projection-valued measure on a compact set $K$, let $\alpha_1$, \dots, $\alpha_N$ be Borel subsets of $K$ such that the intersection of any $m+1$ subsets $\alpha_i$ is empty . Then $$ \sum_{j=1}^N\|E(\alpha_j)\xi\|^2\le m\|\xi\|^2 $$ for any vector $\xi$. \end{lemma} \begin{proof} Since $E(\alpha_j)=\int_K \chi_{\alpha_j}(t)\,dE(t)$, where $\chi_{\alpha_j}$ is the characteristic function of $\alpha_j$, we have that $$ \sum_{j=1}^NE(\alpha_j)=\int_K\Bigl(\sum_j\chi_{\alpha_j}(t)\Bigr)\,dE(t)\le m\int_KdE(t)=mI. $$ Therefore, $\sum_{j=1}^N\|E(\alpha_j)\xi\|^2=\sum_{j=1}^N(E(\alpha_j)\xi,\xi) \le m\,\|\xi\|^2$. \end{proof} \begin{theorem} \label{cth2} Let one of the following conditions hold\textup: \begin{itemize} \item[\textup{(a)}] $g_i$, $i=1,\ldots,n$, are polynomials\textup; \item[\textup{(b)}] $g_i\in\Lip\sigma(N)$, $i=1$,\dots, $n$, and the Hausdorff dimension of $\sigma(M)$ is less then $2$. \end{itemize} Then ${\mathfrak M}(\Gamma)=\ker\Delta$. \end{theorem} \begin{proof} Let $B\in {\mathfrak M}(\Gamma)$, then $\supp_{M,N}B\subset S$. For any $\beta\subset \sigma(N)$ we will denote the set $\pr_1(\pr_2^{-1}(\beta))$ by $\tilde{\beta}$. It is easy to show that if $\beta$ is closed then so is $\tilde \beta$, and that $\pr_1(\pr_2^{-1}(\cup_{k=1}^{\infty}\beta_k))=\cup_{k=1}^{\infty}\tilde {\beta_k}$, for any sequence $\{\beta_k\}$. Hence $\tilde{\beta}$ is a Borel set for any ${F}_{\sigma}$-set $\beta$ (here we consider only such $\beta$). Since $(\sigma(M)\setminus\tilde\beta)\times \sigma(N)$ does not intersect $S$, one has $(I-E_M(\tilde{\beta}))\,BE_N(\beta)=0$. Let $\varepsilon>0$ and let $\displaystyle\sigma(N)=\cup_{j=1}^K\alpha_j$, where $\alpha_i\cap\alpha_j=\emptyset$, $\diam\alpha_j<\varepsilon$, and $K=K(\varepsilon)$ is the least possible for all such decompositions. Then \begin{equation} \Delta(B)=\sum_{j=1}^K\Delta(B)E_N(\alpha_j)= \sum_{j=1}^KE_M(\tilde\alpha_j)\,\Delta(B)E_N(\alpha_j). \end{equation} Set $\gamma=\bigl\{t\in \sigma(M)\colon\sum_{i=1}^nf_i(t)\,g_i(N)=0 \bigr\}$. Then \begin{equation}\label{p21} E_M(\gamma)\Delta=0. \end{equation} In fact, for any $Y\in L(H_N, H_M)$, we have $$ E_M(\gamma)\Delta(Y)=\sum_{i=1}^nE_M(\gamma)\,f_i(M)\,Yg_i(N), $$ and \begin{align*} &(E_M(\gamma)\Delta(Y)\xi, \nu)=\sum_{i=1}^n(Yg_i(N)\xi, E_M(\gamma)\,(f_i(M))^*\nu) \\ &\qquad{}=\sum_{i=1}^n\int_{\gamma}\bigl(Yg_i(N)\xi, \overline{f_i(t)}\,dE_M(t)\nu\bigr) \\ &\qquad=\int_{\gamma}\Bigl(\sum_{i=1}^nYg_i(N)\,f_i(t) \xi,dE_M(t)\nu\Bigr)=0, \end{align*} for any $\xi\in H_N$, $\nu\in H_M$. Relation (\ref{p21}) implies \[ \Delta (B)=\sum_{j=1}^KE_M(\hat{\alpha}_j)\,\Delta (B)\,E_N(\alpha_j), \] where $\hat{\alpha}_j=\tilde\alpha_j\setminus\gamma$. Hence, \begin{gather} |(\Delta(B)\xi, \nu)|=\Bigl|\sum_{j=1}^K\bigl(E_M(\hat{\alpha}_j) \Delta (B)E_N(\alpha_j)\xi, E_M(\hat{\alpha}_j)\nu\bigr)\Bigr| \notag \\ \le\Bigl(\sum_{j=1}^K\bigl\|E_M(\hat{\alpha}_j)\Delta (B)E_N(\alpha_j)\xi\bigr\|^2\Bigr)^{1/2} \Bigl(\sum_{j=1}^K\|E_M(\hat{\alpha}_j)\nu \|^2\Bigr)^{1/2}.\label{p22} \end{gather} Since $\hat{\alpha}_j\subset \pr_1((\hat\alpha_j\times\alpha_j)\cap S)$, we can apply Lemma~\ref{l1} to the operators $NE_N(\alpha_j)$, $ME_M(\hat{\alpha}_j)$. We obtain \begin{gather*} \|E_M(\hat{\alpha}_j)\Delta (B)E_N(\alpha_j)\|\le C\|B\|\diam\alpha_j \le C\|B\|\,\varepsilon, \\[3pt] \|E_M(\hat{\alpha}_j)\Delta (B)E_N(\alpha_j)\xi\|\le C\|B\|\,\varepsilon \|E_N(\alpha_j)\xi\|, \\ \sum_{j=1}^K\|E_M(\hat{\alpha}_j)\Delta (B)E_N(\alpha_j)\xi\|^2 \le C^2\|B\|^2\varepsilon^2\|\xi \|^2, \end{gather*} since the sets $\alpha_j$ are mutually disjoint. First let all $g_i(\cdot)$ be polynomials and let $m$ be the greatest degree of the polynomials $g_i(\cdot)$. Then, for any $t\in\sigma(M)\setminus\gamma$, the equation $\Phi(t,s)=0$ does not have more then $m$ roots. Thus, $t$ can not belong to more than $m$ sets $\hat{\alpha}_j$, because for any $t\in \hat{\alpha}_j$ there exists $s(t)\in \alpha_j$ such that $\Phi(t,s(t))=0$, and the sets $\alpha_j$ are disjoint. Applying Lemma~\ref{l2}, we can conclude that $\sum_{j=1}^K\|E_M(\hat{\alpha}_j)\nu\|^2\le m\|\nu\|^2$. Hence, $|(\Delta (B)\xi, \nu)|\le C\|B\|\,\varepsilon \|\xi\|\,m^{1/2}\|\nu \|^2$. Letting $\varepsilon\rightarrow 0$, we obtain $\Delta (B)=0$. Now assume that the second condition holds. We can estimate the second factor in the right-hand side of (\ref{p22}) as follows: $$ \Bigl(\sum_{j=1}^K\|E_M(\hat{\alpha}_j)\nu\|^2\Bigr)^{1/2}\le (K\|\nu \|^2)^{1/2}= K^{1/2}\|\nu \|. $$ Thus $\|\Delta(B)\|\le C\|B\|\,\varepsilon K^{1/2}$ ($K=K(\varepsilon)$). Since $\dim \sigma(M)<2$, $K(\varepsilon)=o(\varepsilon^{-2})$, this implies $\Delta(B)=0$. \end{proof} \begin{remark}\rm The restriction on $\dim\sigma(M)$ in Theorem~\ref{cth2} can be removed (see \cite{143}), but the proof becomes much more complicated, so we omit this generalization here. \end{remark} \medskip\noindent\textbf{5.} As a corollary from Theorem~\ref{cth2} we obtain the following Kleinecke--Shirokov type theorems. \begin{theorem} \label{c1} If two polynomial semilinear relations have the same graph, then their representations coincide. \end{theorem} \begin{example} By Theorem~\ref{c1}, the semilinear relations $(\ad_{A,q})^nB=0$ and $\ad_{A,q}B=AB-qBA=0$ have the same representations by the bounded operators $A=A^*$, $B$, since the characteristic functions $\Phi(t,s)=t-qs$ and $\Phi(t,s)=(t-qs)^n$, corresponding to the first and the second relations respectively, define the same graph. In particular, for $q=1$ we obtain that bounded operators $A=A^*$, $B$ satisfy the relation $(\ad_A)^nB=[A,\dots[A,B]\dots]=0$ if and only if $A$, $B$ commute, and for $q=-1$, a pair of bounded operators $A=A^*$, $B$ is a solution of the equation $(\ad_{A,-1})^nB=\{A,\ldots\{A,B\}\ldots\}=0$ iff $AB=-BA$. \end{example} \noindent\textbf{6.} As corollaries from Theorem~\ref{cth2} we will also give some results, where the form of the characteristic binary relation corresponding to the semilinear relation (\ref{hom}) is more important then the form of the functions $f_i$, $g_i$. \begin{theorem}\label{c2} Let $f$, $g$, be bounded Borel functions. A pair of boun\-ded operators $A=A^*$, $B$ satisfies the equation $$ f(A)B=Bg(A) $$ if and only if $\supp_A(B)\subset\Gamma$, where $\Gamma=\{(t,s)\mid f(t)=g(s)\}$. \end{theorem} \begin{proof} Set $M=f(A)$, $N=g(A)$. Since $E_M(\alpha)=E_A(f^{-1}(\alpha))$ for any Borel set $\alpha$, the condition $\supp_A(B)\subset\Gamma$ is equivalent to the following one $$ \supp_{M,N}(B)\subset\{(t,s)\mid t=s\}. $$ Theorem~\ref{cth2} gives the required statement. \end{proof} In a similar way one shows that Theorem~\ref{cth2} implies the Fuglede--Putnam--Rozenblum theorem see, for example~\cite{137} and also Section~\ref{sec:1.4.3}. In particular, if bounded operators $A=A^*$, $B$ satisfy the relation $AB=qBA$, where $|q|=1$, then $AB=\bar qBA$, since the sets $\{(t,s)\in{\mathbb R}\times{\mathbb R}\mid t=qs\}$ and $\{(t,s)\in{\mathbb R}\times{\mathbb R}\mid t=\overline qs\}$ coincide. \begin{remark}\rm Sets of the form $\{(t,s)\mid f(t)=g(s)\}$, where $f$, $g$ are bounded Borel functions are called rectangular sets. For them, a much stronger result is true: an operator supported by a rectangular set $F$ satisfies any relation whose characteristic binary relation contains $F$~\cite{143}. The proof is complicated and we restrict ourselves to a more particular result. \end{remark} \begin{theorem}\label{gr} Let $F$ be the graph of a function, that is, $F=\{(t,s)\mid s=\varphi(t)\}$, where $\varphi$ is a bounded Borel function. If $\supp _A(B)\subset F$, then the pair $A$, $B$ gives a representation of any relation whose binary relation contains $F$. \end{theorem} \begin{proof} Let $F\subset\Gamma=\bigl\{(t,s)\mid\sum_{i=1}^nf_i(t)g_i(s)=0\bigr\}$. Since $$ \supp_A(B)\subset \{(t,s)\mid s=\varphi(t)\}\subset \{(t,s)\mid g_i(s)= g_i(\varphi(t))\}, $$ we have $Bg_i(A)=g_i(\varphi(A))B$, $i=1$, \dots, $n$, by Theorem~\ref{c2}. On the other hand, $$ \sum_{i=1}^nf_i(t)\,g_i(\varphi(t))=0 $$ for any $t\in \sigma(A)$, hence $\sum_{i=1}^nf_i(A)\,g_i(\varphi(A))=0$. It follows that \par\vskip\abovedisplayskip\hfill$\displaystyle \sum_{i=1}^nf_i(A)Bg_i(A)=\sum_{i=1}^nf_i(A)(Bg_i(A)-g_i(\varphi(A))B)=0. $\hfill \end{proof} \begin{corollary}\label{c4} Let $A=A^*$, and let there exist a decomposition of $\sigma(A)$ into Borel sets $P_i$, $\sigma(A)=\cup_{i}P_i$, such that each $P_i\times P_j\cap\Gamma$, $i$, $j=1$, $2$,~\dots, is the graph of a function. Then $\ker \Delta_A={\mathfrak M}_A(\Gamma)$. \end{corollary} \begin{proof} Theorem~\ref{gr} implies $E_A(P_i)\Delta (B)E_A(P_j)=0$ for any $i,j$ and hence $\Delta (B)=0$.\end{proof} \begin{remark} Let us note that if\/ $\Gamma$ is such that the set $\{\mu\in {\mathbb R}\mid (\lambda, \mu)\in \Gamma\}$ is finite or countable for any $\lambda \in {\mathbb R}$, then the condition of Corollary~\ref{c4} is satisfied. \end{remark} \medskip\noindent\textbf{7.} As was noticed, the converse of\/ Theorem~\ref{th2-semi} is false in the general case. Here we construct an example which simultaneously solves the Fuglede--Weiss problem. Namely, we show that there exist a pair of bounded operators $A=A^*$, $B$ and continuous functions $f_i$, $g_i$ such that $$ \sum_{i=1}^mf_i(A)Bg_i(A)=0,\quad \text{but} \quad \sum_{i=1}^m\bar{f}_i(A)B\,\bar{g}_i(A)\ne 0. $$ \begin{example}\rm\label{ex} Let $D({\mathbb R}^n)$ be the space of compactly supported infinitely differentiable functions on ${\mathbb R} ^n$, $D'({\mathbb R}^n)$ be its dual space, $FL^1({\mathbb R} ^n)$ the Fourier algebra, $PM({\mathbb R} ^n)$ the space dual to $FL^1({\mathbb R} ^n)$ (the space of pseudo-measures), $F$ the Fourier transform, $\varphi*\psi$ the convolution of two functions in $D({\mathbb R} ^n)$, $\tilde \varphi(x)=\overline{\varphi(-x)}$. Consider the following polynomial in six variables $$ p(x_1,\ldots, x_6)=x_1^2+x_2^2+x_3^2-1+i(x_4^2+x_5^2+x_6^2-1). $$ Let $s_i$, $r_i$, $i=1$, \dots, $m$, be polynomials satisfying the relation $$ p(x-y)=\sum_{i=1}^ms_i(x)\,r_i(y) $$ for any $x,y\in {\mathbb R} ^6$. Let $u,v\in D({\mathbb R} ^n)$ and $a_i=us_i$, $b_i=vr_i$. Obviously, $a_i$, $b_i\in D({\mathbb R}^6)$. Consider the operators $A_i=M_{a_i}$, $B_i=M_{b_i}$, $i=1$, \dots, $m$ in the space $L^2({\mathbb R} ^6)$ (here $M_f$ is the operator of multiplication by the function $f$). Since the Fourier transform of a pseudo-measure $\Phi$ belongs to $L^{\infty}({\mathbb R} ^6)$, the operator $T=F^{-1}M_{F\Phi}F$ is well defined in the space $L^2({\mathbb R}^6)$. Furthermore a direct computation shows that \begin{equation} \Bigl(\sum_{i=1}^mM_{b_i}TM_{a_i}\varphi,\psi\Bigr)=\langle p\,\Phi,u\varphi* \widetilde{\bar{v}\psi}\rangle \end{equation} for any $u$, $v$, $\varphi$, $\psi\in D({\mathbb R}^6)$, where $\langle\cdot,\cdot\rangle$ is the pairing of the spaces $D({\mathbb R}^6)$, $D'({\mathbb R}^6)$ (since $PM({\mathbb R} ^6)\subset D({\mathbb R} ^6)'$, $p\,\Phi\in D'({\mathbb R}^6)$ for any polynomial $p$ in six variables). Since the set $$ L=\{u\varphi*\tilde{\bar{v}\psi}\mid \varphi, \psi,u,v\in D({\mathbb R}^6)\} $$ is dense in $D({\mathbb R}^6)$, the existence of a pseudo-measure $\Phi$ such that $p\,\Phi=0$ and $\bar{p}\,\Phi\ne0$ would imply the existence of functions $u$, $v$ such that \begin{align*} \Bigl(\sum_{i=1}^mM^*_{b_i}TM^*_{a_i}\varphi, \psi\Bigr)&= \langle\bar{p}\,\Phi, \bar{u}\varphi*\widetilde{v\psi}\rangle\ne 0, \\*[-8pt] &\qquad\qquad \text{for some $\varphi$, $\psi\in D({\mathbb R}^6)$}, \\ \Bigl(\sum_{i=1}^nM_{b_i}TM_{a_i}\varphi,\psi\Bigr)&= \langle p\,\Phi,u\varphi*\widetilde {\bar {v}\psi}\rangle=0, \\*[-8pt] &\qquad\qquad \text{for all $\varphi$, $\psi\in D({\mathbb R}^6).$} \end{align*} Hence, for the operator $A_i$, $B_i$, $T$ constructed relatively to $\Phi$, $u$, $v$, we obtain \begin{equation}\label{prp} \sum_{i=1}^mB_iTA_i=0 \quad \text{and} \quad \sum_{i=1}^mB_i^*TA_i^*\ne 0. \end{equation} Since any finite commutative family of normal operators can be realized as a family of continuous functions of one self-adjoint operator, one can find a self-adjoint operator $A$ and continuous functions $f_i$, $g_i$ such that $B_i=f_i(A)$, $A_i=g_i(A)$. Then (\ref{prp}) is equivalent to the following $$ \sum_{i=1}^mf_i(A)\,T\,g_i(A)=0 \quad\text{and}\quad \sum_{i=1}^m\bar{f}_i(A)\,T\,\bar{g}_i(A)\ne 0. $$ Thus, it remains to prove the existence of a pseudo-measure $\Phi$ such that $p\,\Phi=0$ and $\bar{p}\,\Phi\ne 0$. In order to do this, we consider the $\delta$\nobreakdash-mea\-sure $\mu_0$ of the surface $S^2$ in ${\mathbb R} ^3$. Let $\mu=\mu_0\times\mu_0$. It is easy to show that $$ |F\mu(\lambda)|=\Bigl|\int_{{\mathbb R} ^6}e^{i(\lambda,x)}\,d\mu(x)\Bigr|=O(|\lambda |^{-1}), $$ as $|\lambda |\rightarrow +\infty$ in ${\mathbb R} ^6$. Since $F(\frac{\partial}{\partial x_i}f)=\lambda _iFf$, the Fourier transform $F(\frac{\partial\mu}{\partial x_i})$ belongs to $L^{\infty}({\mathbb R} ^6)$, so that $\frac{\partial\mu}{\partial x_i}\in PM({\mathbb R} ^6)$. The pseudo-measure $\frac{\partial\mu}{\partial x_i}$ has a compact support. Thus $L\mu\in PM({\mathbb R} ^6)$ for any first-order differential operator $L$ with smooth coefficients. Let $$ L=(1+i)\,x_4\frac{\partial}{\partial x_1}-(1-i)\,x_1\frac{\partial}{\partial x_4}, \quad\text{and} \quad \Phi=L\mu. $$ It is easy to see that $Lp=0$ and $L\bar{p}=4(1+i)\,x_1x_4$, which implies that for any function $\varphi$ in $D({\mathbb R}^6)$, \begin{align*} \langle p\, \Phi,\varphi\rangle &=\langle L\mu,p\varphi\rangle= \langle \mu,L(p\varphi)\rangle =\langle\mu,pL(\varphi)\rangle=0, \\ \langle\bar{p}\,\Phi,\varphi\rangle&=\langle \mu,L(\bar{p})\varphi\rangle + \langle\mu, \bar{p}L \varphi\rangle= 4(1+i)\langle x_1x_4\,\mu,\varphi\rangle. \end{align*} Thus, $p\,\Phi=0$ and $\bar p\,\Phi=4(1+i)\,x_1x_4\,\mu\ne 0.$ \end{example} \begin{remark} Instead of one operator $A=A^*$ one can consider a family of commuting self-adjoint operator operators ${\mathbf A}=(A_k)_{k=1}^m$ which are connected with a bounded operator $B$ by a semilinear relation of the form \begin{equation}\label{mul} \sum_{i=1}^n f_i({\mathbf A})\,B\,g_i({\mathbf A})=0, \end{equation} where $f_i({\mathbf A})$, $g_i({\mathbf A})$ are Borel bounded functions of the family ${\mathbf A}$. As before, one can associate the set $$ \Gamma=\{(t,s)\in {\mathbb R} ^m\times{\mathbb R} ^m \mid \sum_{i=1}^n f_i(t)g_i(s)=0\} $$ to relation \eqref{mul}. Support of the operator $B$ ($\supp_{\mathbf A}B\subset \sigma ({\mathbf A})\times\sigma({\mathbf A})$) with respect to the family ${\mathbf A}$ is defined in the same way as in Definition~\ref{def1} but with the joint resolution of the identity $E_{\mathbf A}(\cdot)$ for the family ${\mathbf A}$ instead of $E_{M}(\cdot)$ and $E_{N}(\cdot)$. If ${\mathfrak M}_{\mathbb A}(\Gamma)$ is the set of all bounded operators whose support belongs to $\Gamma$ and $\ker\Delta_{\mathbb A}$ is the operator of multiplication, $X\mapsto \sum_{i=1}^n f_i({\mathbf A})\,B\, g_i({\mathbf A})$, then using similar arguments to those given in Section~\ref{sec:1.3.3}, item~4, one can prove that $\ker\Delta_{\mathbf A}\subset{\mathfrak M}_{\mathbf A}(\Gamma)$. But the converse inclusion is not true already for $m$ greater than 2 and polynomials $f_k$, $g_k$. The corresponding example can be derived using similar arguments to those given in Example~\ref{ex}. \end{remark} \begin{remark} As has been shown above, an operator $B$, $A$-supported by the set $\Gamma=\{(t,s)\mid\sum_{i=1}^nf_i(t)\,g_i(s)=0\}$, might not satisfy the corresponding semilinear relation $\sum_{i=1}^nf_i(A)B\,g_i(A)=0$ if the functions $f_i(\cdot)$, $g_i(\cdot)$ are not smooth. But this is true if the operator $B$ is pseudo-integral. Let us recall the definition of a pseudo-integral operator. Let $\mu$ be the scalar spectral measure of a self-adjoint operator $A$, let $H=\int_{\sigma(A)}^\oplus l_2(N(\lambda))\,d\mu(\lambda)$, $N(\lambda)\in{\bf N} \cup\{\infty\})$, be the spectral resolution of $A$, and let $m$ be a bounded measure on ${\mathbb R} \times {\mathbb R} $ such that the projections of $m$ on both coordinates are majorized by $\mu$. The measure $m$ is said to be regular. Since $l_2(N)$ for $N\in {\mathbb N}$ can be embedded into $l_2(\infty)$, we can define the bilinear form $$ (\vec{x}, \vec{y})\mapsto \iint (\vec{x}(s),\vec{y}(t))\,dm(t,s), $$ where $(\vec{x}(s),\vec{y}(t))$ is the scalar product in $l_2(\infty)$. If $m$ is regular, then the bilinear form defines some bounded operator $T_m$. The operator $T_m$ is called a ``pseudo-integral operator constructed relatively to $m$''. It was shown by Arveson that the pseudo-integral operator $T_m$ is supported by any pseudo-closed set on which the measure $m$ is concentrated (the set $E\subset{\mathbb R} \times{\mathbb R} $ is called pseudo-closed if its complement is a union of countably many measurable sets of the form $X\times Y\subset{\mathbb R} \times{\mathbb R} $). Let $\Gamma$ be the characteristic binary relation corresponding to the semilinear relation (\ref{hom}), and $m$ a regular measure with $\supp m\subset\Gamma$. Then, $A$, $T_m$ is a representation of relation (\ref{hom}). Indeed, \begin{align*} &\bigl(\sum_{i=1}^nf_i(A)\,T_m\,g_i(A) \vec{x},\vec{y}\Bigr)=\sum_{i=1}^n\bigl(T_mg_i(A) \vec{ x}, \overline{f_i}(A)\vec{y}\bigr) \\ &\qquad{}=\sum_{i=1}^n\iint_{\Gamma} g_i(s)\,f_i(t)\,(\vec{x}(s),\vec{y}(t))\,dm(t,s) \\ &\qquad{} =\iint_{\Gamma}\Phi(t,s)\,(\vec{x}(s),\vec{y}(t))\,dm(t,s)=0. \end{align*} \end{remark} \subsection{Irreducible representations of semilinear relations} \label{sec:1.3.4} Now we study irreducible representations of semilinear relations, i.e., an irreducible families of operators $A=A^*$, $B$, $B^*$ satisfying $$ \sum_{i=1}^nf_i(A)\,B\,g_i(A)=0. $$ It is clear that if $f_i$, $g_i$ are polynomials, then such a family of operators defines an irreducible representation of the $*$-algebra generated by $a=a^*$, $b$, $b^*$ and the relation $\sum_{i=1}^nf_i(a)\,b\,g_i(a)=0$. In what follows we shall mean by a representation of the semilinear relation a triple $(A=A^*, B, B^*)$ satisfying the semilinear relation instead of the pair ($A=A^*, B$), if it does not lead to any confusion. \medskip\noindent\textbf{1.} We begin with finite-dimensional re\-pre\-sen\-ta\-tions and establish a connection between irreducible representations of \eqref{hom} and the corresponding graph. \begin{proposition}\label{connec} If a family $(A, B, B^*)$ defines a finite-dimensional irreducible representation of \eqref{hom}, then the corresponding graph $\Gamma\restriction_{\sigma(A)}$ is connected. For every finite connected subgraph $(D, \Gamma\restriction_{D})$, there exists an irreducible representation $(A, B, B^*)$ with $\sigma(A)=D$. \end{proposition} \begin{proof} In fact, if $\Gamma\restriction_{\sigma(A)}=\Gamma_1\cup\Gamma_2$ is the union of two connected subgraphs then, by Theorem~\ref{th1-semi}, the spectral subspaces $A$ corresponding to the vertices of\, $\Gamma_1$ and $\Gamma_2$ are invariant with respect to $B$ and $B^*$. This shows that $(A, B, B^*)$ is reducible. Let $\Gamma\restriction_{\sigma(A)}$ be connected. The family of operators $A$, $B$, $B^*$ defined by \begin{align*} A&= \begin{pmatrix} \lambda _1 & & 0\\ & \ddots & \\ 0 & & \lambda _m \end{pmatrix}, \qquad \{\lambda_1,\dots,\lambda_m\}=D, \\[-5pt] &\qquad\qquad\qquad\qquad\qquad\qquad \lambda _i\ne \lambda _j, \qquad i\ne j,\\[3pt] B&=(b_{ij})_{i,j=1}^m, \qquad b_{ij}= \begin{cases} 0, & ( \lambda_i,\lambda_j)\notin \Gamma|_D,\\ 1, & ( \lambda_i,\lambda_j)\in \Gamma|_D, \end{cases} \end{align*} is irreducible. Indeed, the relation $[C,A]=0$ implies that $C$ is diagonal, $C=\diag(c_1,\ldots,c_2)$. From $[C,B]=0$, it follows that $c_kb_{kl}=b_{kl}c_l$. Since $\Gamma\restriction_{\sigma(A)}$ is connected, we have that there exists a permutation $(l_1,\ldots,l_m)\in S_m$ such that $(\lambda_{l_k},\lambda_{l_{k+1}})\in\Gamma\restriction_{\sigma(A)}$, hence $b_{l_k,l_{k+1}}=1$. This implies that $c_1=\ldots=c_m$, and $C=c_1I$. Moreover, we have $\sigma(A)=D$. \end{proof} We also give a reformulation of this statement for the symmetrical case. \begin{proposition} If\/ $A=A^*$, $B=B^*$ is an irreducible finite-dimen\-sional representation of \eqref{hom}, then the graph $\Gamma_s\restriction_{\sigma(A)}$ is connected. For every finite connected subgraph $(D, \Gamma_s\restriction_{\sigma(A)})$, there exists an irreducible pair $A=A^*$, $B=B^*$ satisfying \eqref{hom} such that $D=\sigma(A)$. \end{proposition} The proof is the same as the one given above, but with $\Gamma_s$ replacing $\Gamma$. Note that the constructed operator $B$ is self-adjoint because the graph $\Gamma_s$ is symmetrical. \medskip\noindent\textbf{2.} In what follows, we investigate irreducible representations of \eqref{hom} on a Hilbert space $H$, where $H$ is not necessary finite-dimen\-sional. First, we prove an analogue of the theorem on connectedness of the graph (Proposition~\ref{connec}). For this purpose, we recall some definitions. \begin{definition} A subset $F\subset{\mathbb R}\times {\mathbb R}$ is called marginally null with respect to a measure $\mu$ \textup(or a class of measures equivalent to $\mu$\textup) if $F\subset (\alpha\times {\mathbb R})\cup({\mathbb R}\times \beta)$, where $\mu(\alpha)= \mu(\beta)=0$. \end{definition} Let, further, \begin{align*} \Gamma(M)&=\{y\mid\exists x\in M\colon (x,y)\in \Gamma\}, \\ \Gamma^{-1}(M)&=\{x\mid\exists y\in M\colon (x,y)\in \Gamma\}, \end{align*} and let $M^{c}$ be the complement of the set $M$. In what follows we shall assume that $\Gamma(M)$ and $\Gamma^{-1}(M)$ are Borel for any Borel set $M$. The concepts defined below generalize those of quasi-invariance and ergodicity of a measure. For graphs related to dynamical systems, we will also discuss these notions in Section~\ref{sec:2.1.1}. \begin{definition} We call a set $M\subset{\mathbb R}$ right-invariant \textup(left-invariant\textup) with respect to $\Gamma$ and a measure $\mu$, if the set $M^{c}\times M\cap\Gamma$ \textup(respectively $M\times M^{c}\cap\Gamma$\textup) is marginally null\textup; $M$ is invariant if it is left- and right-invariant. A measure $\mu$ is called \textup(left-, right-\textup) quasi-invariant with respect to $\Gamma$ if the set $\Gamma(M)\cup\Gamma^{-1}(M)$ \textup(respectively $\Gamma^{-1}(M)$, $\Gamma(M)$\textup) is of non-zero measure for every Borel set $M$ such that $\mu(M)\ne 0$, $\Gamma(M)$, $\Gamma^{-1}(M)\in {\mathfrak B}({\mathbb R})$. A spectral type $\mu$ is called \textup(left-, right\nobreakdash-\textup) ergodic with respect to $\Gamma$ if any \textup(left-, right-\textup) invariant set is a $\mu$\nobreakdash-null set or has the $\mu$-null complement. \end{definition} Note that for any operator $A=A^*$ there exists the trivial representation $(A,0,0)$ of the relation (\ref{hom}). Therefore, it is natural to consider representations without trivial parts. \begin{definition} We call a representation $A$, $B$, $B^*$ $*$-complete if for any non-zero spectral subspace $W$ of the operator $A$ one of the operators $B$ and $B^*$ is not equal to zero on $W$. \end{definition} It is easy to show that any irreducible representation of dimension greater than one is $*$-complete. \begin{proposition}\label{pr4} If a representation $(A, B, B^*)$ of \eqref{hom} is $*$-comp\-lete, then the spectral measure of the operator $A$ is quasi-invariant with respect to $\Gamma$. \end{proposition} \begin{proof} If $E_A(M)\ne 0$ but $E_A(\Gamma^{-1}(M))=0$ and $E_A(\Gamma(M))=0$, then $$ BE_A(M)= E_A(\Gamma^{-1}(M))BE_A(M)=0 $$ and $$ B^*E_A(M)=E_A(\Gamma(M))B^*E_A(M)=0, $$ hence $B\restriction E_A(M)H=0$ and $B^*\restriction E_A(M)H=0$. Thus, the representation $(A, B, B^*)$ is not $*$-complete. \end{proof} One can also show that if the spectral measure of $A$ is quasi-invariant, then there exists a bounded operator $B$ such that the triple $(A, B, B^*)$ is a $*$-complete representation of \eqref{hom}. But we omit the proof here. \begin{proposition}\label{therg} If a representation $(A, B, B^*)$ of \eqref{hom} is irreducible, then the spectral measure of $A$ is ergodic with respect to~$\Gamma$. \end{proposition} \begin{proof} Let $(A, B, B^*)$ be an irreducible representation of the relation \eqref{hom}. If the spectral type of $A$ is not ergodic, then there exists $M\subset{\mathbb R}$ such that $E_A(M)\ne 0$, $E_A(M)\ne I$, and the sets $\Gamma\cap(M^{c}\times M)$, $\Gamma\cap(M\times M^c)$ are marginally null sets. Hence, $\Gamma\cap (M^{c}\times M)\subset(M_1\times{\mathbb R})\cup({\mathbb R}\times M_2)$, where $\mu(M_1)=\mu(M_2)=0$. From this it follows that $E_A(M^{c})BE_A(M)= E_A(M^{c}\setminus M_1)BE_A(M\setminus M_2)=0$, i.e., the spectral subspace $E_A(M)H$ is invariant with respect to $A$, $B$. In the same way one can show that $E_A(M)H$ is invariant with respect to $A$, $B^*$. This contradicts the irreducibility of $A$, $B$, $B^*$. \end{proof} Note that the converse statement is also true. Namely, for any ergodic measure $\mu$ there exists an irreducible representation $A$, $B$, $B^*$ of relation (\ref{hom}) such that $\mu$ is a spectral scalar measure of $A$. However we will not discuss the proof here. \medskip\noindent\textbf{3.} Let $(A, B)$ satisfy the following semilinear relation: \begin{equation}\label{mk} AB=BF(A), \end{equation} where $F(\cdot)$ is a bounded Borel mapping defined on ${\mathbb R}$. It is easy to show that in this case the invariance of a set $\Delta\in {\mathfrak B}({\mathbb R})$ with respect to $\Gamma$ and a measure $\mu$ means its invariance with respect to the mapping $F(\cdot)$, i.e., $\mu(F(\Delta))=\mu(\Delta)$ and $\mu(F^{-1}(\Delta))=\mu(\Delta)$. A measure $\mu$ is quasi-invariant with respect to $\Gamma$ if the measures $\mu(F(\cdot))$ and $\mu(F^{-1}(\cdot))$ are absolutely continuous with respect to $\mu$. The ergodicity of the spectral measure with respect to $\Gamma$ means ergodicity of the measure $E_A(\cdot)$ with respect to $F(\cdot)$ (i.e., for any $F(\cdot)$-invariant Borel set $\Delta\subset{\mathbb R}$, either $E_A(\Delta)=0$ or $E_A(\Delta)=I$). Therefore, by Proposition~\ref{therg}, we have that, if $(A, B, B^*)$ is an irreducible representation of \eqref{mk}, then $E_A(\cdot)$ is ergodic with respect to $F(\cdot)$. \medskip\noindent\textbf{4.} In the analysis of the representations of \eqref{mk}, the behavior of the dynamical system $F\colon\mathbb{R} \to \mathbb{R}$ plays a central role. The structure of representations of \eqref{mk} depends on the structure of the orbits of the corresponding dynamical system (see Section~\ref{sec:2.1.1} for details). Here we give the corresponding results for general semilinear relations. We begin with the following definition. \begin{definition} A set $E$ is called\/ $\Gamma$-invariant if $E\times E^c\cap\Gamma=\emptyset$ and $E^c\times E\cap\Gamma=\emptyset$. A minimal $\Gamma\restriction_M$-invariant set $O_M(E)$ containing the set $E$ is said to be a trajectory \textup(semi-trajectory\textup) of the set $E\subset M$ with respect to $\Gamma\restriction_M$. \end{definition} The concept of a trajectory generalizes that of an orbit for dynamical systems. The following result is an analogue of the theorem on the connectedness of the graph supporting an irreducible finite-dimensional representation. \begin{theorem} Let $M$ be a compact set. \textup(a\textup) If there is $x\in M$ such that the trajectory $O_M(\{x\})$ is dense in $M$, then $M$ is the spectrum of an irreducible representation, i.e., there exists an irreducible representation $(A, B, B^*)$ such that $\sigma(A)=M$. \textup(b\textup) If the set $M$ is the spectrum of an irreducible representation of \eqref{hom}, then the trajectory $O_M(G\cap M)$ is dense in $M$ for any open set $G$, where $G\cap M\ne\emptyset$. \end{theorem} \begin{proof} Let $O_M(\{x\})$ be dense in $M$. Then there is a sequence $\{\lambda_n\}_{n=1}^{\infty}\subset O_M(\{x\})$ which is dense in $M$. Now let $\mu $ be a measure concentrated on $\{\lambda_n\}_{n=1}^{\infty}$, and $\mu(\lambda_n)\ne 0$ for any $\lambda_n$. Then $\mu$ is ergodic. In fact, let $S$ be invariant with respect to $\Gamma$, and $\mu(S)\ne 0$, $\mu(S^c)\ne 0$. Then $S\cap\{\lambda_n\}_{n=1}^{\infty}\ne\emptyset$. From this it follows that $S\cap O_M(\{x\})\ne \emptyset$. If $x\in S$, then $O_M(\{x\})\subset S$, hence $\mu(S^c)=0$, which contradicts the assumption. Assume that $x\not\in S$, hence $x\in S^c$. Since $S$ is invariant with respect to $\Gamma$, so is the complement of $S$ and $O_M(\{x\})\cap S^c\ne\emptyset$. From the condition $x\in O_M(\{x\})\cap S^{c}$ we obtain $O_M(\{x\})\cap S^c=O_M(\{x\})$. Hence $O_M(\{x\})\subset S^c$, and $\mu(S)=0$, which gives a contradiction. \medskip\noindent\textbf{5.} Let $G$ be an open set, $G\cap M\ne \emptyset$, $\overline{O_M(G\cap M)}\ne M$, and $E_A(\cdot)$ be the spectral measure of the operator $A$ with spectrum $M$. Then $E_A(G)\ne 0$ and there exists a compact set $F\subset G\cap M$, $E_A(F)\ne 0$. The set $O_M(F)$ is a Borel set. Since $\overline{O_M(F)}\ne M$, $E_A(O_M(F))\ne I$. Thus, $E_A(\cdot)$ is not ergodic with respect to $\Gamma\restriction_M$, because $O_M(F)$ is an invariant set. \end{proof} \begin{definition} A set $\tau$ is said to be a measurable section of $(D, \Gamma)$ if $\tau\subset D$ is a Borel set and every trajectory $O_D(\{x\})$ with respect to $\Gamma\restriction_D$ intersects $\tau$ at exactly one point. \end{definition} \begin{proposition} Let $(D, \Gamma)$ have a measurable section. Then, for any irreducible representation of \eqref{hom} with $\sigma(A)\subset D$, there exists a unique trajectory $O_D(\{x\})$ for which $E_A(O_D(\{x\}))=I$. \end{proposition} \begin{proof} Let $\tau$ be a measurable section of $(D, \Gamma)$. Assume that $\mu $ is not concentrated on any orbit $O_D(\{x\})$. We prove that there exists a partitions of $\tau$ into two sets $\tau=\tau_1\cup\tau_2$, $\tau_1\cap\tau_2=\emptyset$, such that $\mu(O_D(\tau_1))>0$, $\mu(O_D(\tau_2))>0$. Consider the trajectories $T_i=O_D^i(\tau_i)$, $i=1$, $2$, for any two sets $\tau_i$ satisfying the conditions $\tau=\tau_1\cup\tau_2$, $\tau_1\cap\tau_2=\emptyset$. By the definition of a measurable section, $T_1\cap T_2=\emptyset$ and $T_1\cup T_2=D$. Assuming that for any such decomposition $D=T_1\cup T_2$ one of the values $\mu(T_1)$ and $\mu(T_2)$ is equal to zero, we can find a decreasing sequence $\{T^k\}$ such that $E_A(T^k)=I$ for any $k$ and $\bigcap T^k=O_D(\{x\})$ for some $x\in\tau$. From the latter argument we can conclude that $\mu$ is concentrated on an orbit which contradicts the assumption. Let $\tau=\tau_1\cup\tau_2$ be the required decomposition, i.e. $\mu(O_D(\tau_i))>0$, $i=1,2$. Since both sets $O_D(\tau_1)$ and $O_D(\tau_2)$ are invariant with respect to $\Gamma|_D$, the existence of the decomposition implies a contradiction to ergodicity of the measure $\mu$. Thus, $\mu$ is concentrated on the trajectory of some point $x$. \end{proof} If there is no measurable section for the graph ($D$, $\Gamma$) of the semilinear relation \eqref{hom}, then the structure of representations with bounded operators $(A, B, B^*)$ is more complicated: there might exist irreducible representations such that the spectrum of the operator $A$ is not discrete. \begin{remark} The same theorems are valid for the representation with $B=B^*$, but with $\Gamma_s$ instead of $\Gamma$. \end{remark} \subsection{Representations of semilinear ${F}_4$-relations} \label{sec:1.3.5} The problem of describing all irreducible representations of semilinear relations up to unitary equivalence might be very difficult. The complexity of the description depends on the structure of the corresponding graph. \begin{theorem} $1$. If all connected components of the graph $\Gamma$ corresponding to a semilinear relation are of the form: $\onepoint {}$, $ \edgeright {}{}$, then any irreducible representation $(A, B, B^*)$ of the relation is one- or two-dimensional\textup: \begin{itemize} \item[\textup(i\textup)] $A=\lambda$, $B=0$, $\lambda\in{\mathbb R}$\textup; \item[\textup(ii\textup)] $A=\bigl( \begin{smallmatrix} \lambda_1&0\\ 0&\lambda_2 \end{smallmatrix}\bigr)$, $B=\bigl( \begin{smallmatrix} 0&0\\ b&0 \end{smallmatrix}\bigr)$, where $(\lambda_1, \lambda_2, b)$ belongs to the set $K_1=\bigl\{(\lambda_1, \lambda_2, b)\in {\mathbb R}^3\mid \Phi(\lambda_1,\lambda_2)=0;\,\lambda_1\ne\lambda_2, \,b>0\bigr\}$. \end{itemize} $2$. If all connected components of the graph $\Gamma_s$ are of the form $\onepoint {}$, $\ear{}$, $\edge {}{} $, then any irreducible representation $(A, B=B^*)$ of the relation is one- or two-dimensional\textup: \begin{itemize} \item[\textup(i\textup)] $A=\lambda$, $B=0$, $\lambda\in{\mathbb R}$\textup; \item[\textup(ii\textup)] $A=\lambda$, $B=b$, where $(\lambda, b)$ belongs to the set $K_2=\{(\lambda, b)\in {\mathbb R}^2\mid \Phi(\lambda,\lambda)=0\}$\textup; \item[\textup(iii\textup)] $A=\bigl( \begin{smallmatrix} \lambda_1&0\\ 0&\lambda_2 \end{smallmatrix}\bigr)$, $B=\bigr( \begin{smallmatrix} 0&b\\ b&0 \end{smallmatrix}\bigr)$, where $(\lambda_1, \lambda_2, b)$ belongs to $K_3=\bigl\{(\lambda_1, \lambda_2, b)\in {\mathbb R}^3\mid \Phi(\lambda_1,\lambda_2)=\Phi(\lambda_2,\lambda_1)=0;\, \lambda_1\ne \lambda_2,\, b>0\bigr\}$. \end{itemize} \end{theorem} Note that if $f_i$, $g_i$ are polynomials, then the algebra ${\mathbb C}\langle a,b\mid \sum_{i=1}^nf_i(a)\,b\,g_i(a)=0\rangle$ is an ${ F}_4$-algebra. \begin{proof} Under the above condition we have that $\Gamma$ is the graph of a bijective mapping $\phi\colon N_1\rightarrow N_2$, where $N_1\cap N_2=\emptyset$ (i.e., $\Gamma=\{(t,s)\in N_2\times N_1\mid t=\phi(s)\}$), and $\Gamma_s$ is the graph of a bijective mapping $\phi\colon N\rightarrow N$ (i.e., $\Gamma=\{(t,s)\in N^2\mid t=\phi(s)\}$). Moreover, $\phi$ is measurable under the assumption that $\Gamma(M)$ and $\Gamma^{-1}(M)$ are Borel sets for any Borel $M$. Therefore, by Corollary~\ref{cth2}, if $(A,B)\in L(H)$ is a representation of a relation with such a graph, then ($A$, $B$) satisfy the relation $AB=B\phi(A)$, and the spectrum of the operator $A$ belongs to $N_1\cup N_2$ in the first case, and to $N$ in the second one. Clearly, the operators $A$, $BB^*$ commute. Let us consider the first case. We have $B^2=0$ and, therefore, $\ker B\ne 0$. Denote the subspace $ \ker B\cap\ker B^*$ by $H_1$. It is easy to show that $H_1$ is invariant with respect to $A$, $B$, $B^*$, and all irreducible representations defined on $H_1$ are one-dimensional and given by $(i)$. Now assume that $\ker B\cap\ker B^*=\emptyset$. Set $H_0=\ker B\cap(\ker B^*)^{\perp}$. Since $H_0\subset E_A(\phi(N_1))H$, $B^*H_0\subset E_A(N_1)H$, and $N_1\cap\phi(N_1)=\emptyset$, the subspaces $H_0$, $B^*H_0$ are orthogonal. Moreover, $H_0$, $B^*H_0$ are invariant with respect to $A$, $BB^*$, which imply that given $\Delta\in {\mathfrak B}({\mathbb R}^2)$, the subspace $E(\Delta)H_0\oplus B^*E(\Delta)H_0$ is invariant with respect to $A$, $B$, $B^*$, where $E(\cdot)$ is the joint resolution of the identity for the commuting pair of operators $A$, $BB^*$ restricted to $H_0$. From this it follows that $\Delta$ is concentrated in one point if $A$, $B$, $B^*$ is an irreducible family. Thus for such a family of operators there exists a joint eigenvector $e\in H_0$ for $A$, $BB^*$, and $\{e,B^*e\}$ define an orthogonal basis of the representation space. The corresponding irreducible representation is given by $(ii)$. In the case $A=A^*$, $B$ we have that the operators $A\phi(A)$, $A+\phi(A)$ commute with $A$, $B$, and hence due to the irreducibility, they are multiples of the identity, i.e., $A\phi(A)=a_1I$ and $A+\phi(A)=a_2I$. Then $A^2-a_2A+a_1I=0$, and so the spectrum of $A$ is $\sigma(A)=\{\lambda_1,\lambda_2\}$, where $\lambda_1$, $\lambda_2$ are the roots of the equation $\lambda^2-a_2\lambda+a_1=0$. Hence the spectrum of $A$ is discrete as soon as ($A$, $B$) is irreducible. In addition, $B^2$ commutes with $A$, $B$ and is a multiple of the identity, $B^2=b^2I$. If $b\ne 0$ and $e_{\lambda_1}$ is an eigenvector of $A$, then $e_{\lambda_1}$, $Be_{\lambda_1}$ define an invariant subspace; moreover $Be_{\lambda_1}$ is an eigenvector with the eigenvalue $\phi(\lambda_1)=:\lambda_2$. Therefore, by the irreducibility, the operators $A$, $B$ can be at most two-dimensional. If $\phi(\lambda_1)\ne\lambda_1$ (i.e., $\Phi(\lambda_1,\lambda_1)\ne 0$), then normalizing the orthogonal basis $e_{\lambda_1}$, $Be_{\lambda_1}$ we get an orthogonal basis in which operators $A$, $B$ are of the form $(iii)$. For $\phi(\lambda)=\lambda$ one has that $A$, $B$ commute, and hence we can choose a joint eigenvector $e_{\lambda,b}$, which define an invariant subspace. From this one can conclude that the corresponding irreducible representation is one dimensional and given by $(i)$ or $(ii)$. \end{proof} If the graph $\Gamma$ corresponding to a semilinear relation contains the subgraphs: $\ear {}$ or $\twovec $ (and with any other orientation), and the graph $\Gamma_s$ contains the subgraphs: $\earedge{}{} $, or $\twoedge{}{}{} $, then the problem of describing all irreducible representations $(A, B, B^*)$ and $(A, B=B^*)$ respectively becomes very complicated (the corresponding $*$-algebra is wild). We refer the reader to Sections~\ref{sec:3.1.1} and \ref{sec:3.1.2} for a precise definition of $*$-wild algebras, and to Section~\ref{sec:3.1.4} for the proof of the above fact. %%% Local Variables: %%% mode: latex %%% TeX-master: "the" %%% TeX-master: "the" %%% End: