Spectral Analysis of Random Graphs - Semantic Scholar
arXiv:1512.03489v1 [math.ST] 11 Dec 2015
Moment-Based Spectral Analysis of Random Graphs with Given Expected Degrees
Victor M. Preciado†
&
M. Amin Rahimian
Department of Electrical and Systems Engineering, University of Pennsylvania (e-mail: [email protected])
In this paper, we analyze the limiting spectral distribution of the adjacency matrix of a random graph ensemble, proposed by Chung and Lu, in which a given expected degree sequence T
(n)
(n)
wn = (w1 , . . . , wn ) is prescribed on the ensemble. Let ai,j = 1 if there is an edge between the nodes {i, j} and zero other-
wise, and consider the normalized random adjacency matrix √ of the graph ensemble: An = [ai,j / n]ni,j=1 . The empirical spectral distribution of An denoted by Fn (·) is the empirical measure putting a mass 1/n at each of the n real eigenvalues of the symmetric matrix An . Under some technical conditions on the expected degrees sequence, we show that with probability one, Fn (·) converges weakly to a deterministic distribution F (·). Furthermore, we fully characterize this distribution by providing explicit expressions for the moments of F (·).
Keywords: Random Matrix Theory, Random Graph Models, Graph Spectrum
†
This work was supported by the National Science Foundation grants CNS-1302222 “NeTS: Medium: Collaborative Research: Optimal Communication for Faster Sensor Network Coordination”, and IIS1447470 “BIGDATA: Spectral Analysis and Control of Evolving Large Scale Networks”.
Spectral Analysis of Random Graphs
1
1. Introduction Understanding the relationship between structural and spectral properties of a network is a key question in the field of Network Science. Spectral graph methods (see [1, 2, 3], and references therein) have become a fundamental tool in the analysis of large complex networks, and related disciplines, with a broad range of applications in machine learning, data mining, web search and ranking, scientific computing, and computer vision. Studying the relationship between the structure of a graph and its eigenvalues is the central topic in the field of algebraic graph theory [4, 1, 2, 3, 5]. In particular, the eigenvalues of matrices representing the graph structure, such as the adjacency or the Laplacian matrices, have a direct connection to the behavior of several networked dynamical processes, such as spreading processes [6], synchronization of oscillators [7], random walks [8], consensus dynamics [9], and a wide variety of distributed algorithms [10]. The availability of massive databases describing a great variety of real-world networks allows researchers to explore their structural properties with great detail. Statistical analysis of empirical data has unveiled the existence of multiple common patterns in a large variety of network properties, such as power-law degree distributions [11], or the small-world phenomenon [12]. Random graphs models are the tool-of-choice to analyze the connection between structural and spectral network properties. Aiming to replicate empirical observations, a variety of synthetic network models has been proposed in the literature [11, 12]. The structural property that have (arguably) attracted the most attention is the degree distribution. Empirical studies show that the degree distribution of important real-world networks, such as the Internet [13], Facebook, or Twitter, are heavy-tailed and can be approximated using power-law distributions [14, 15]. We find in the literature that several random graphs models are able to model degree distributions from empirical data. One of best known models is the configuration model, originally proposed by Bender and Canfield in [16]. This model is able to fit a given degree sequence exactly (under certain technical conditions). Although many structural properties of this model have been studied in depth [17, 18], this model is not specially amenable to spectral analysis. A tractable alternative to the preferential attachment model was proposed by Chung and Lu in [19], and analyzed in [20, 21, 22]. In this model, which we refer to as the Chung-Lu model, an expected degrees sequence is prescribed onto a random graph ensemble, that can be algebraically described using a (random) adjacency matrix. Studies of the statistical properties of the eigenvalues of random graphs and networks are prevalent in many applied areas. Examples include the investigations of the spacing between nearest eigenvalues in random models [23, 24], as well as real-word networks [25, 26, 27]. Empirical observations highlight spectral features not observed in classical random matrix ensembles, such as a triangular like eigenvalue distribution in power-law networks [28, 29] or an exponential decay in the tails of the eigenvalue distribution [30, 31]. 1.1. Main Contributions In this work we offer an exact characterization for the eigenvalue spectrum of the adjacency matrix of the random graph models with a given expected degree sequence.
2
V. M. Preciado and M. A. Rahimian
This characterization is in terms of the moments of the eigenvalue distributions and it hinges upon application of the moments method for identifying the limiting spectral distribution of a related sequence of random matrices that is constructed from centralizing and normalizing the graph adjacencies. Accordingly, we give closed form expressions that describe the almost sure limits of the spectral moments and can be used to draw important conclusions about the graph spectrum and to bound several quantities of interest relating to the eigenvalue distribution of the graph adjacencies. The remainder of this paper is organized as follows. Preliminaries on the background and motivation of our study as well as the random graph model that we consider are presented in Section 2. Our main results on the asymptotic spectral moments of the adjacencies of random graphs and the characterization of the limiting spectral distributions for the normalized adjacencies are presented in Section 3, where we also include an outline of the proofs which are presented in detail in the Mathematical appendix at the end. In Section 4, we apply our results to a case where node degrees are obtained by random samples from the support of a preset function and show how our main results can be applied in analysis of the spectrum of large random graphs. Section 5 concludes the paper.
2. Background & Motivation 2.1. Chung-Lu Random Graph Model We consider the Chung-Lu random graph model introduced in [19] and analyzed in [20, 21, 22], in which an expected degree sequence given by the n non-negative entries of T (n) (n) the vector wn = (w1 , . . . , wn ) is prescribed over the nodes of the graph ensemble.1 In this model, each edge of the random (undirected) graph on n vertices (labeled by [n]) is realized independently of all other edges and in accordance with the probability measure P{·}. Let E {·} and Var {·} be the expectation and variance operators corresponding to P{·}. Following [32] we allow for self-loops. To each random graph we associate a (random) adjacency matrix, which is a zero-one matrix with the (i, j)-th entry being one if, and only if, there is an edge connecting nodes i and j. The number of edges incident to a vertex is the degree of that vertex, and by the volume of a graph we mean the sum of the degrees of its vertices. In this paper, our primary interest is in characterizing the asymptotic behavior of the eigenvalues of the random adjacency matrix as the graph size n increases. We consider the distribution of these eigenvalues over the real line and characterize this distribution through its moments sequence. Accordingly, the probability of there Pn (n) (n) (n) being an edge between nodes i and j is equal to ρn wi wj , where ρn = 1/ i=1 wi is the inverse expected volume. The adjacency relations in this random graph model are (n) √ represented by an n × n real-valued, symmetric random matrix An = [aij / n], where 1
Throughout this paper, R and N are the set of real and natural numbers, N0 = {0} ∪ N, n ∈ N is a parameter denoting the size of the random graph, [n] denotes {1, 2, . . . , n}, and card(X ) denotes the cardinality of set X . The n × n identity matrix is denoted by In , random variables are printed in boldface, matrices are denoted by capital letters. Every vector is marked by a bar over its lower case letter.
Spectral Analysis of Random Graphs
3
n o (n) (n) (n) (n) aij are independent zero-one random variables with E aij = ρn wi wj . As we shall √ see in the sequel, the 1/ n normalization is such that the distribution of the eigenvalues of the normalized adjacency An converges almost surely to a deterministic distribution that is uniquely characterized by its sequence of moments. As a main result of this paper, we explicate the technical conditions under which this convergence property holds true (cf. Assumptions 1 to 4 below). Furthermore, we proffer explicit expressions for calculating this moments sequence. These expressions, in turn allow us to upper or lower bound various quantities of interest pertaining to the spectrum of the random graph adjacencies (cf. Section 4 and discussions therein). Characterization of the convergence condition for the moments depends critically on the behavior of the extreme values in the expected degree sequence with the increasing size n. To that end, we consider two sequences {w ˆn : n ∈ N} and {w ˇn : n ∈ N} given (n) (n) by w ˆn = maxi∈[n] wi and w ˇn = mini∈[n] wi for all n ∈ N. Another quantity of interest whose evolution with the graph size n plays an important role in the preceding arguments is the second-order average degree d˜n given by [20], Pn (n) 2 n 2 X i=1 wi (n) = ρ wi = ρn kwn k22 . d˜n = Pn n (n) i=1 i=1 wi T
For our main results to hold true we need the expected degree sequence wn to satisfy the following conditions. Assumption Assumption Assumption Assumption
1 2 3 4
(Sparse and Graphical): ρn w ˆn2 < 1, ∀n and ρn w ˆn2 = o(1). (Logarithmic Growth): w ˆ n /w ˇn = O(log n). (Eigenvector Concentration): d˜n /ρn = o(n3 / log n). √ (Vanishing Effect of Centralization): d˜n = O( n log n).
2.2. Spectrum of the Chung-Lu Random Graph In 2003 by a series of results, Chung, Lu and Vu established important asymptotic properties of the spectra of the adjacency matrices of random graph models with given expected degree sequence [20, 21, 22]. A key result of theirs specifies the almost sure limit of the largest eigenvalue of the adjacency matrix as follows2 [20, Theorems 2.1 and 2.2]. √ Theorem 2.1 (Largest Eigenvalue of Random Graphs). If d˜n > w ˆn log n, then √ with probability one the largest eigenvalue of unnormalized adjacency, λn ( nAn ), is √ (1 + o(1))d˜n ; while if w ˆn > d˜n log2 n, then then the largest eigenvalue is almost surely √ (1 + o(1)) w ˆn .
2
Given three functions f (·), g(·), and h(·) we use the asymptotic notations f (n) = O(g(n)) and f (n) = o(h(n)) to signify the relations lim supn→∞ |f (n)/g(n)| < ∞ and limn→∞ |f (n)/h(n)| = 0, and we use f (n) (g(n)) to mean that f (n) = (1 + o(1))g(n).
4
V. M. Preciado and M. A. Rahimian
In particular, depending on the exponent of the power-law for a random graph model whose expected degree distribution obeys a power-law, either of the two regimes can be the case; and regardless, the largest eigenvalue is always with probability one less √ √ than 7 log n max{ w ˆn , d˜n }. In [20] similar conditions are established for the almost sure convergence of the k-th largest eigenvalues to the square root of the k-th largest expected degree, and [22] also considers relevant results concerning the spectra of other graph associated matrices such as the Laplacian. More recent results use the machinery of concentration inequalities to investigate the behavior of the graph spectra for random graphs with independent edges [33, 34, 35]. In [35] the author shows concentration of the spectral radius norm for the Laplacian and Adjacency matrices around their expectations. These results are improved by Chung and Radcliffe [33] who use a Chernoff type inequality and approximate the eigenvalues by those of the expected matrices. The √ ˆn log n) and it approximation bound in [33] for the eigenvalues of the adjacency is O( w √ ˆn by [34]. is later improved to (2 + o(1)) w 3. Main Result Here, we offer a moment-based characterization of the limiting distribution of the eigenvalues of the Chung-Lu random graph model given Assumptions 1 to 4 on the expected degree sequence. The moments sequence provides a versatile tool in the spectral analysis √ of complex networks [36, 37, 38]. It is worth highlighting that in the 1/ n normalization regime that we work, as n → ∞ the largest eigenvalue escapes to infinity. This is a common case when investigating the limiting distribution of a sequence of distributions that some mass escapes to infinity, which can cause the limit distribution to be not a probability distribution (does not integrate to one), in which case the underlying sequence of distribution is not “tight”. However, as we show in this paper, the tightness property holds for the sequence of spectral distributions in the Chung-Lu random graph model. It is because the mass that is escaping to infinity (a single largest eigenvalue) is asymptotically vanishing itself: as n → ∞, the contribution of a single eigenvalue to the continuous √ limiting spectral distribution in the 1/ n normalization regime is vanishingly small. By the same token, our results complement the characterization of the largest eigenvalue √ by [20, 21, 22]; which focuses on the adjacency matrix itself, as opposed to the (1/ n)normalized version, and investigates the growth rate and concentration of the largest eigenvalue as described in Subsection 2.2. To describe our main results we need to introduce some terminology. Let λ1 (An ) ≤ λ2 (An ) ≤ . . . ≤ λn (An ) be n real-valued random variables representing the n eigenvalues of the random matrix An ordered from the smallest to the largest. We define Ln {·} = Pn (1/n) i=1 δλi (An ) {·}, as the random probability measure on the real line that assigns mass uniformly to each of the n eigenvalues of the random matrix An . Here, δx {·} is the probability measure on R assigning unit mass to point x ∈ R and zero elsewhere. The corresponding distribution Fn (x) = Ln {(−∞, x]} = (1/n)card({i ∈ [n] : λi (An ) ≤ x}), is a random variable for each x ∈ R and is referred to as the empirical spectral distri(n) bution for the random matrix An . Moreover, we define mk = (1/n)trace(Akn ) R +∞ (ESD) k = −∞ x d Fn (x) as the real-valued random variable which is the k-th spectral moment
Spectral Analysis of Random Graphs
5
of the random matrix An . Our main results constitute in concluding the almost sure and weak convergence of the empirical spectral distribution of the adjacency An to a deterministic distribution F (·). We call this distribution the limitingR spectral distribution +∞ (LSD) and we characterize it through its moments sequence, mk = −∞ xk dF (x). These moments are derived in terms of the limiting average power sums of the normalized degree sequence defined as follows. √ (n) For each i ∈ N, we define the limiting normalized degrees σi = limn→∞ ρn wi . The limiting average of the k-th power sum of the sequence {σi , i ∈ N} is given by Λk = Pn limn→∞ (1/n) i=1 σik . Note from Assumption 1 that σi < 1 for all 1, which in turn implies that Λk < 1 for all k. According to our main result, the spectral moments of the limiting spectral distribution F (·) are specified as follows.
Theorem 3.1 (Spectral Moments of Random Graphs). Consider the random graph model with given expected degree sequence wn , satisfying Assumptions 1 to 4. With probability one, Fn (·) converges weakly to F (·) and F (·) is the unique distribution function R +∞ satisfying ∀k ∈ N, −∞ xk dF (x) = mk where for all s ∈ N0 , m2s+1 = 0, and X 2 s+1 (3.1) Λr1 Λr2 ...Λrss , m2s = s + 1 r1 , ..., rs 1 2 r∈Rs
with r = (r1 , r2 , ..., rs ), Rs = {r ∈ Ns0 :
Ps
j=1 rj
= s + 1,
Ps
j=1
j rj = 2s}.
3.1. Proof Outline The crux of the argument is in showing that for each k ∈ N, the k-th spectral moments (n) mk converge almost surely to mk , thence concluding by the method of moments that with probability one the empirical spectral distributions Fn (·) converge weakly to F (·). A centralization idea would allow us to proceed in parallel with the proof steps in our earlier results in [39], concerning random matrices with independent zero-mean entries and rank-one pattern of variances. To begin, we consider the centralized version of the adjacency An given by i h √ (n) ˆ n = An − E {An } = (1/ n)ˆ aij . (3.2) A (n)
Note that the entries ˆ aij have zero means and asymptotically rank one pattern of variances given by o 2 n (n) (n) (n) (n) (n) (n) Var ˆ aij = E ˆ aij = ρn wi wj (1 − ρn wi wj ) σi σj , (3.3) (n)
(n)
(n)
since by Assumption 1, ρn wi wj = o(1). Furthermore, ˆ aij entries are all uniformly n o (n) almost surely bounded: P |ˆ aij | ≤ 1 = 1. Indeed, except for the fact that the pattern ˆ n is asymptotically rank one rather than exactly rank of variances for the entries of A ˆ one, the random matrix An together with the sequence {σi , i ∈ N}, satisfy the all four necessary conditions of the rank one ensemble investigated in [39]. It is therefore to be
6
V. M. Preciado and M. A. Rahimian
expected that save for a few minor refinements, the moment-based analysis of the random ˆ n just as well. matrix ensemble in [39] applies to A To proceed we introduce some necessary notations. Similarly to An , we consider the ˆ n ordered from the smallest to the largest as λ1 (A ˆ n ) ≤ λ 2 (A ˆ n) ≤ . . . ≤ eigenvalues of A (n) k ˆ ˆ λn (An ) and define the random variable m ˆ k = (1/n)trace(An ) to be its k-th spectral (n) (n) moment. Also let m ¯ k = E{m ˆ k } be the expected spectral moments for all k, n. The proof of our main result proceeds as follows. (n) (n) Lemma A1.2 of the Appendix ensures that under Assumptions 3 and 4, m ˆ k mk , almost surely for each k ∈ N. Therefore, the effect of centralization on the spectral moments is asymptotically vanishing. Next Lemma A2.2 provides the asymptoticly exact (n) expressions: m ¯ k mk for the expected spectral moments under Assumptions 1 and 2. Lemma A2.1 implies that the same expressions in fact describe the almost sure limits (n) of the spectral moments: m ˆ k mk for all k ∈ N. The latter, together with the earlier (n) results, imply that under Assumptions 1 to 4: mk mk , almost surely for all k ∈ N. This pointwise almost sure convergence of the moments sequence would in turn imply the weak convergence of the empirical distributions Fn (·) to the deterministic distribution F (·) characterized uniquely by the sequence of limiting spectral moments. This fact is established in Lemma A3.1, completing the proof. 3.2. The Case of Erd˝ os-R´ enyi Random Graphs In Erd˝os-R´enyi random graphs denominated popularly as Gn,p each edge is realized with a probability p and independently of all else. This is a special case of the random graph model with given expected degrees, where the expected degrees are fixed and the expected degree sequence wn is equal to (np, np, . . . , np)T . Ever since its introduction in the late 1950s by Erd˝os and R´enyi [40, 41], properties of this well-known class of random graphs have been extensively studied and they continue to attract attention [42, 43]. Indeed, the seminal work of F¨ uredi and Koml´ os [44] immediately gives important asymptotic properties of the spectra of Erd˝ os-R´enyi Random Graphs; with probability one, putting its largest eigenvalue at (1 + o(1))np and upper-bounding the absolute values of the rest p by (2 + o(1)) np(1 − p). More recently, Feige and Ofek [45] have shown that under mild √ conditions on p, the largest eigenvalue of adjacency is almost surely pn + O( pn) and √ all other eigenvalues are almost surely O( pn). Fig. 1(a) depicts the histogram of eigenvalues for a particular realization of the un√ normalized adjacency nAn , with n = 1000 nodes and p = 0.05. In particular, we can √ observe that the largest eigenvalue λ1 ( nAn ) is located away from the remaining smaller ˆ n defined in (3.2). We plot a ones. Let us also consider the centralized adjacency matrix A typical realization of the eigenvalue histogram of the unnormalized centralized adjacency √ ˆ nAn in Fig. 1(b). Notice that, as pointed out in Lemma A1.2, the effect of centralizing the adjacency matrix An (w) is to cancel out the largest eigenvalue, pushing it back to zero while the bulk of eigenvalues remains (almost) unperturbed. Subsequently, the √ effect of normalization on the limiting spectral distribution in the 1/ n-normalization regime is vanishing. This can be also noticed in Fig. 1(c), where we plot the empirical ˆ n (x). Indeed under the 1/√n-normalization that we spectral distributions Fn (x) and F
Spectral Analysis of Random Graphs
7
√ investigate the limiting spectral distributions, the largest eigenvalue of An grows as np, √ escaping almost surely to infinity; every other eigenvalue is almost surely O( p), being asymptotically compactly supported.3
Figure 1. (a) Eigenvalue histogram of the Erd˝ os-R´ enyi random graph; (b) eigenvalue histogram of the centralized adjacency; (c) spectral distributions of the normalized adjacency An (blue solid) and the ˆ n (red dashed). centralized version A
4. Degrees Specified by Random Uniform Sampling (n)
Here we consider the case where the expected degree of each node i is specified as wi = fn (xi ), where fn (·) are given functions with a common support normalized to be the unit interval [0, 1], and {xi , i ∈ N} is a random sample, uniformly and independently drawn from the unit interval. To illustrate our results, let us assume that fn (x) = ∆n e−αn x , 0 ≤ x ≤ 1, and consider a Chung-Lu random graph model with the expected degree (n) sequence specified as wi = ∆n e−αn xi , i ∈ [n]. Exponential degree distributions have been observed in practice, and are known to be good descriptors of the structural brain network data that are obtained through diffusion imaging techniques [48, 49]. The almost sure asymptotic expression for the second-order average degree d˜n can be obtained from a Monte Carlo average [50, Section XVI.3], and is given by the strong law of large numbers as follows, Pn (n) 2 R1 2 R 1 2 −2α x n f (x)dx ∆ e dx i=1 wi ∆n (1 − e−2αn ) 0 n ˜ R1 = R01 n = , (4.1) dn = Pn (n) 2(1 − e−αn ) f (x)dx ∆n e−αn x dx i=1 wi 0 n 0 √ ˜ n > ∆n log n, and the and we know from Theorem 2.1 that if ∆n > log2 n, then d √ ˜ n . We now use Theorem largest eigenvalue is almost surely given by λ1 ( nAn ) d 3.1 to write closed-form expressions for the asymptotic spectral moments of An , the normalized adjacency of the random graph whose expected degree sequence is given by 3
It is possible to derive the limiting spectral distribution of this random graph ensemble from Theorem √ 3.1 under Assumptions 1 to 4, provided that p = pn = O(log n/ n). Then w ˆn = w ˇn = npn , ρn = −1 n−2 pn , and d˜n = npn all satisfy the requirements of Assumptions 1 to 4 and Theorem 3.1 gives the s pn 2s , which correspond to a semi circular distribution the asymptotic spectral moments as m2s = s+1 s √ √ supported over [−2 pn , +2 pn ], [46, 47]. Applying the results of F¨ uredi and Koml´ os [44] for the limiting spectral distribution of random matrices with independent zero-mean entries and identical variances provides that the same semi-circular distribution still describes the LSD of centralized and normalized adjacency matrix even when pn = p is fixed and is not decreasing with n.
8
V. M. Preciado and M. A. Rahimian (n)
(n)
wTn = (w1 , . . . , wn ). We begin by calculating the almost sure asymptotic expression of the inverse expected volume as follows, Z 1 Z 1 n X n∆n 1 (n) ∆n e−αn x dx = = fn (x)dx = n 1 − e−αn . (4.2) wi n ρ n i=1 αn 0 0
We now proceed to verify the qualifying conditions for applying Theorem 3.1 to this random graph model. First note the following almost sure asymptotic identities for the maximum and minimum degrees, (n)
∆n ,
(n)
∆n e−αn ,
w ˆ n = max wi i∈[n]
ˇ n = min wi w i∈[n]
(4.3)
both of which can be verified easily from the corresponding order statistics for the uniform √ distribution on the unit interval.4 If we set ∆n = n log n and αn = log log n, then from the set of almost sure asymptotic identities in (4.1), (4.2) and (4.3) we get √ √ ˜ n ∆n /2 = n log n/2 = O( n log n), d w ˆn eαn = log n = O(log n), ˇn w r √ ∆n α n log n log log n √ ρn w ˆn = = o(1), n n1/4 3 ˜n d n∆2n n n2 log2 n =o , = ρn 2αn 2 log log n log n Hence, Assumptions 1 to 4 are all satisfied and we can apply Theorem 3.1 to obtain the closed-form expressions for the limiting spectral moments of the normalized adjacency An . First note that the k-th order limiting averages Λk are almost surely given by Z 1 Z 1 √ k k ( ρn ∆n )k √ √ ρn fn (x) dx = ρn ∆n e−αn x dx = 1 − e−kαn . Λk kαn 0 0
4
More specifically, if we note that mini∈[n] xi and maxi∈[n] xi are respectively distributed as Beta(1, n) and Beta(n, 1) variables [51, Chapter 2], then the claimed almost sure limits follow by the Borel-Cantelli lemma. To see how, consider their expected values: E{mini∈[n] xi } = 1/(n + 1) and E{maxi∈[n] xi } = n/(n + 1), and apply the Chebyshev inequality to their quadratically decaying common variance, Var{maxi∈[n] xi } = Var{mini∈[n] xi } = n/ (n + 1)2 (n + 2) .
Spectral Analysis of Random Graphs
9
Under these conditions, we have the following closed-form expression for the asymptotic spectral moments of An : √ s s X 2 s+1 Y X 2 s+1 Y ( ρn ∆n )krk ρsn ∆2s n m2s = k −rk s+1 s + 1 r1 , ..., rs (kαn )rk s + 1 r1 , ..., rs αn r∈Rs
=
r∈Rs
k=1
k=1
s X 2 s+1 Y logs n k −rk , ns/2 log log n r∈R s + 1 r1 , ..., rs k=1
(4.4)
s
Ps Ps where in the second equality we have used the identities k=1 rk = s + 1 and k=1 krk = 2s. The histogram of the eigenvalues for the normalized adjacency matrix An of a particular realization with parameters ∆ = 10, α = 1, and n = 1000 is plotted in Fig. ˆ n = An − [√ρn w(n) w(n) ]n 2. The largest eigenvalue of the centralized adjacency A i,j=1 in i j ˆ n ) = 5.6214. We can upper and lower bound the largest this realization is given as λn (A eigenvalue using the k-th spectral moments as follows [24, Equation (2.66)]: 1/k ˆ kn )1/k ≤ λn (A ˆ n ) ≤ n · trace(A ˆ kn ) trace(A . (4.5)
If we consider k = 20 in (4.5) and use the asymptotic spectral moment m20 available ˆ ˆ 20 from (4.4) to replace for trace(A n ), then the lower and upper bounds on λn (An ) are as 1/20 1/20 follows: (m20 ) = 4.3193 and (n · m20 ) = 6.1011. These values compare reasonably ˆ with the empirically observed value λn (An ) = 5.6214. Using the techniques in [37], we can formulate semi-definite programs that improve the bounds in in (4.5) by taking into account the knowledge of all spectral moments up to a fixed order, as described in [39, Section 4].
60 50 40 30 20 10 0 Figure 2.
-6
-4
-2
0
2
4
6
Eigenvalue histogram of a sample realization from the random graph model with exponential degree sequence and n = 1000 vertices.
10
V. M. Preciado and M. A. Rahimian 5. Conclusions
In this work, we investigated the asymptotic behavior of the eigenvalues of the adjacency of random graph models with specified expected degrees sequence. We showed that in √ the 1/ n normalization regime that we consider, and under some technical assumptions on the expected degrees sequence, the empirical spectral distribution of the adjacency converges weakly to a deterministic distribution, and we gave closed-form expressions for its limiting spectral moments. We illustrated the application of our results to spectral analysis of large-scale networks with exponential degree distributions. The latter are found to be good descriptors for the structural brain network data, obtained through diffusion imaging. Furthermore, closed-form expressions of the moments allow us to bound various quantities of interest relating to the spectrum such as the spectral radius. These bound can be strengthened using SDP formulations that take into account the knowledge of all spectral moments up to a fixed order.
Spectral Analysis of Random Graphs
11
Appendix: Proof of the Main Result The argument leading to the proof of our main convergence result in Theorem 3.1 is executed in three steps. We begin by showing in Section A1 that with probability (n) (n) ˆn one limn→∞ mk = limn→∞ m ˆ k , i.e. the adjacency An and its centralized version A share the same almost sure limits for their spectral moments. Next in Section A2 we (n) refine our arguments from [39] leading to the almost sure limits limn→∞ m ˆ k = mk , with mk given by (3.1) as in Theorem 3.1. Finally in Section A3 we apply the method of moments to conclude the weak and almost sure convergence of the empirical spectral distributions Fn (·) to the deterministic distribution F (·) determined by the moments sequence mk , k ∈ N that we have obtained earlier. A1. Effect of Centralization on the Spectral Moments The following set of results measures the effect of the centralization in (3.2) by comparing ˆ n and An asymptotically as n → ∞. Indeed, centralization the spectral moments of A by subtracting the mean E {An } from the adjacency An as in (3.2), has the effect of shifting the largest eigenvalue back to zero and symmetrizing the spectrum. Example 1 demonstrates this shifting; however, in the sequel we shall show that the subsequent effect on the spectral moments is asymptotically vanishing provided that certain mild assumptions on the degree sequences are satisfied. √ ˆn log n, then λ1 (An ) d˜n with probability We know from Theorem 2.1 that if d˜n > w one. We use a variation of this result to prove that the vector of expected degrees is asymptotically almost surely an eigenvector of An associated its largest eigenvalue. In particular, as λn (An ) concentrates around d˜n , the vector (1/n)An wn also concentrates 1 ˜ d w . This is important when characterizing the effect of the round the vector n√ n n n centralization in (3.2) in light of the fact that T T ρn ρn d˜n E {An } = √ wi wj = √ wn wn = √ wn wn . (A1.1) 2 n n nkwn k2 i,j∈[n] Lemma A1.1 (Eigenvector Concentration). Under Assumption 3, it is true that 1 1 ˜ √ n An w n n n dn w n almost surely. Proof. The i-th component of An wn is a random variable given by: (n) n X aij (n) √ [An wn ]i = w . n j j=1 (n)
(n)
(A1.2) (n)
(n)
Since aij is a Bernoulli random variable with P{aij = 1} = ρn wi wj , so that we have 2 d˜n (n) (n) ρn w j = √ wi , n j=1 n
(n)
w E {[An wn ]i } = √i
n X
12
V. M. Preciado and M. A. Rahimian
√ (n) (n) √ and E {An wn } = (d˜n / n)wn . Next note that each of the summands aij wj / n in (n) (n) √ (n) √ (A1.2) are independent bounded random variables satisfying aij wj / n ∈ [0, wj / n] almost surely. Hence, we can apply Hoeffding’s inequality [52, Theorem 2 ] to get that for each i and any ε > 0 ( ) 3 2 ˜ dn (n) 1 −2n ε −2n3 ρn ε2 /d˜n . P [An wn ]i − √ wi ≥ ε ≤ 2 exp P 2 = 2e n n (n) n w i i=1
Next note that given d˜n /ρn = o(n3 / log n) per Assumption 3, with ε as in above and for any α > 1, we get that when n is large enough, d˜n /ρn < 2n3 ε2 /(α log n); whence 3 2 ˜ 2e−2n ρn ε /dn < 2/nα forms a summable series in n, and by the Borel-Cantelli lemma [53, Theorem 4.3] we get that ( ) ˜n (n) 1 d P [An wn ]i − √ wi ≥ ε, i.o. = 0, n n n which holds true for any ε > 0, and therefore we have ) ( d˜n (n) 1 = 1. P lim [An wn ]i = lim √ wi n→∞ n n n→∞ n
The claimed concentration of eigenvector around wn now follows by the countable intersections of the above almost sure events over all i ∈ N. We can now proceed to give conditions under which the spectral moments of An and ˆ n are asymptotically almost surely identical, and therefore the effect of centralization A on spectral moments is asymptotically vanishing.
Lemma A1.2 (Vanishing Effect of Centralization). Under Assumptions 3 and 4, (n) (n) m ˆ k mk , almost surely for each k ∈ N. Proof. To begin consider the k = 1 case. From (3.2) we have (n)
=
m ˆ1
1 ˆ n ) = 1 trace(An − E{An }). trace(A n n
From (A1.1) we know that T d˜n d˜n trace(wn wn ) = √ , 2 nkwn k2 n √ ˆ n ) = trace(An ) − d˜n / n, is true for all n, and in wherefrom it follows that trace(A particular with probability one as n → ∞. For general k ∈ N, we have
trace(E{An }) = √
(n)
m ˆk
=
1 ˆ kn ) = 1 trace(An − E{An })k . trace(A n n
Spectral Analysis of Random Graphs
13
To proceed, consider the binomial expansion of (An − E{An })k consisting of a sum of the product of non-commutative elements as follows: k−2 Akn + Ak−1 n (−E{An }) + An (−E{An })An + . . . 2 k−3 + Ak−2 n (−E{An }) + An (−E{An })An (−E{An }) + . . .
Consider any product term of the form, Π(k) = Akn1 (−E{An })k2 Akn3 . . . (−E{An })kp ,
(A1.3)
where k = (k1 , . . . , kp ) is an integer partition of k consisting of p > 1 positive integers satisfying k1 + . . . + kp−1 = k. Let k˜ be the sum of evenly indexed integers: k˜ = k2 + k4 + . . . + k2·bp/2c . We claim that as n → ∞, d˜kn 1 ˜ trace(Π(k)) → (k/2)+1 (−1)k , n n
(A1.4) 2(ki −1)
T
almost surely. To see why, first note that for any ki , (wn wn )ki = kwn k2 taking the ki -th power of both sides in (A1.1), we get (−E{An })ki =
T
wn wn , so
T (−d˜n )ki wn wn , 2 k /2 i n kwn k2
and replacing in (A1.3) yields Π(k) =
(−d˜n ) ˜ nk/2
˜ k
T
Akn1 (
T
wn wn wn wn )Ak3 . . . ( ). kwn k22 n kwn k22
(A1.5)
Under Assumption 3, from Lemma A1.1 we know that (1/n)An wn (d˜n /n3/2 )wn almost surely. Multiplying both sides by Akni −1 for any integer ki yields 1 ki d˜kni An wn (k /2)+1 wn , n n i almost surely. Hence, taking the limits of both sides in (A1.5) we get T 1 d˜nk wn wn ˜ Π(k) (−1)k (k/2)+1 ( ), n kwn k22 n T
almost surely. Taking the trace of both sides and using the fact that trace(wn wn /kwn k22 ) = 1, leads to (A1.4) as claimed. The above applies invariably to any of the product terms appearing in the binomial expansion of (An − E{An })k , with the exception of the leading term, trace(Akn ), which does not simplify any further. Next note that given any 1 ≤ k˜ ≤ k, the number of terms Π(k) in the binomial expansion for which (A1.4) holds true, is exactly kk˜ ; wherefore we get that with probability one, k ˜k X 1 1 d k ˜ ˆ kn ) = trace(An − E{An })k trace(Akn ) + n trace(A (−1)k ˜ . n n k nk/2 ˜ k=1
14
V. M. Preciado and M. A. Rahimian
We next use a trite identity, 0 = (1 − 1)k = 1 + 1 ˆ kn ) 1 trace(A n n
Pk
(−1) ˜ k=1
trace(Akn ) −
almost surely. The claim now follows upon noting that √ is true because d˜n = O( n log n) per Assumption 4.
˜ k k ˜ k
d˜nk nk/2
!
to get ,
1 ˜ √ k n (dn / n)
= o(1), ∀k ∈ N, which
A2. Almost Sure Limits of the Spectral Moments In this section we shall establish that the asymptotic relations for the almost sure limits ˆ n are indeed given by (3.1) in of the spectral moments of the centralized adjacency A (n) Theorem 3.1, i.e. we have m ˆ k mk , almost surely for each k ∈ N. The underlying arguments parallel those in [39] and some details are omitted here in the interest of expositional brevity and with no peril to the formality of the arguments, as the omitted details are replicas of those in [39]. In particular, Lemma A3.13 in [39] on the almost sure convergence of the spectral moments holds true in the case of our random matrix ˆ n and with no change to the proof steps therein. Accordingly, we have: ensemble A Lemma A2.1 (Almost Sure Convergence of the Spectral Moments). For any (n) (n) k ∈ N, if m ¯ k mk , then m ˆ k mk almost surely. ˆ n it Hence, for investigating the almost sure convergence of the spectral moments of A suffices to consider their expectations in the limit. This is achieved by first identifying the terms that asymptotically dominate the behavior of each moment of any order as in Subsection 3.1 of [39], and then deriving the asymptotically exact expressions for each of the identified terms following [39, Subsection 3.2]. The analysis leading to Theorem ˆ n , provided that we 3.4 and Theorem 3.5 of [39] hold true verbatim in the case of A √ √ ˆn and ρn w ˇn respectively. By the same token, replace σ ˆn and σ ˇn therein, by ρn w ˆ n , establishing Theorem 3.6 in [39] can be recycled for our random matrix ensemble A (3.1) as the asymptotically exact expressions for the even moments. Here, in addition √ (n) to replacing σ ˆn and σ ˇn , we should also change every σi term in that proof to ρn wi for finite n; however, this adjustment does not influence the conclusion of the theorem which concerns the limit as n → ∞ of the dominant term identified by Theorem 3.4, since 2 √ (n) (n) we have defined σi = limn→∞ ρn wi , ∀i and per (3.3), E ˆ aij σi σj . Thereby, combining the results of Theorem 3.4, Theorem 3.5 and 3.6 in [39] and after the minor adjustments indicated above, we obtain: (n)
Lemma A2.2 (Limiting Spectral Moments). Under Assumptions 1 and 2, m ¯ 2s (n) m2s and m ¯ 2s−1 = o(1), for all s ∈ N.
Spectral Analysis of Random Graphs
15
A3. Weak Convergence of the Spectral Distribution The results of Sections A1 and A2 thus far enable us to claim that under Assumptions (n) 1 to 4, the spectral moments sequence {mk , k ∈ N} converges pointwise almost surely to the deterministic sequence {mk , k ∈ N}. Coup de grˆ ace is to conclude the almost sure and weak convergence of the empirical spectral distributions from the pointwise almost sure convergence of their moments sequence; it is the province of the method of moments whose details are spelled out in [39, Appendix A2] and they can be reused word for word in the case of our empirical spectral distributions Fn (·): (n)
Lemma A3.1 (Method of Moments, Theorem 3.1 of [39]). If mk mk almost surely for all k ∈ N, then with probability one Fn (·) converges weakly R +∞to F (·) as n → ∞, where F (·) is the unique distribution function satisfying ∀k ∈ N, −∞ xk dF (x) = mk .
In fact, Theorem 3.1 of [39] provides more and it is further true that Fn (·) converges weakly, almost surely, to F (·) as n → ∞ (cf. [39, Appendix A1] for the definitions of the two modes of weak convergence). We have thus pieced together all the ingredients required of our main result characterizing the moment sequence of the almost sure weak √ limit of the (1/ n)-normalized adjacency matrix of random graph models with specified expected degree sequence.
16
V. M. Preciado and M. A. Rahimian
References
[1] F. Chung, Spectral Graph Theory (CBMS Regional Conference Series in Mathematics, No. 92). American Mathematical Society, Dec. 1996. [2] D. M. Cvetkovic, M. Doob, and H. Sachs, Spectra of graphs: Theory and application. Academic Press, New York, 1980, vol. 413. [3] D. Cvetkovi´c, P. Rowlinson, and S. Simi´c, “An introduction to the theory of graph spectra,” Cambridge-New York, 2010. [4] N. Biggs, Algebraic Graph Theory, 2nd ed. Cambridge: Cambridge University Press, 1993. [5] R. Merris, “Laplacian matrices of graphs: a survey,” Linear Algebra and its Applications, vol. 197-198, no. 0, pp. 143 – 176, 1994. [6] P. Van Mieghem, J. Omic, and R. Kooij, “Virus spread in networks,” IEEE/ACM Transactions on Networking, vol. 17, no. 1, pp. 1–14, Feb 2009. [7] L. M. Pecora and T. L. Carroll, “Master stability functions for synchronized coupled systems,” Physics Review Letters, vol. 80, no. 10, pp. 2109–2112, March 1998. [8] L. Lov´ asz, “Random walks on graphs: A survey,” Combinatorics, vol. 2, no. 1, pp. 1–46, 1993. [9] R. Olfati-Saber, “Flocking for multi-agent dynamic systems: Algorithms and theory,” IEEE Transactions on Automatic Control, vol. 51, no. 3, pp. 401–420, 2006. [10] N. A. Lynch, Distributed algorithms. Morgan Kaufmann, 1996. [11] A. Barab´ asi and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, no. 5439, pp. 509–512, 1999. [12] D. Watts and S. Strogatz, “Collective dynamics of small-world networks,” Nature, no. 393, pp. 440–442, 1998. [13] M. Faloutsos, P. Faloutsos, and C. Faloutsos, “On power-law relationships of the Internet topology,” in SIGCOMM, 1999, pp. 251–262. [14] A. Clauset, C. R. Shalizi, and M. E. J. Newman, “Power-law distributions in empirical data,” SIAM Review, vol. 51, no. 4, pp. 661–703, 2009. [15] A.-L. Barab´ asi and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, no. 5439, pp. 509–512, 1999. [16] E. A. Bender and E. R. Canfield, “The asymptotic number of labeled graphs with given degree sequences,” Journal of Combinatorial Theory, Series A, vol. 24, no. 3, pp. 296–307, 1978. [17] M. Molloy and B. Reed, “The size of the giant component of a random graph with a given degree sequence,” Combinatorics, Probability and Computing, vol. 7, no. 03, pp. 295–305, 1998. [18] S. Chatterjee, P. Diaconis, and A. Sly, “Random graphs with a given degree sequence,” The Annals of Applied Probability, vol. 21, no. 4, pp. 1400–1435, 2011. [19] F. Chung and L. Lu, “Connected components in random graphs with given expected degree sequences,” Annals of combinatorics, vol. 6, no. 2, pp. 125–145, 2002. [20] F. Chung, L. Lu, and V. Vu, “Eigenvalues of random power law graphs,” Annals of Combinatorics, vol. 7, no. 1, pp. 21–33, 2003.
Spectral Analysis of Random Graphs
17
[21] ——, “Spectra of random graphs with given expected degrees,” Proceedings of the National Academy of Sciences, vol. 100, no. 11, pp. 6313–6318, 2003. [22] ——, “The spectra of random graphs with given expected degrees,” Internet Mathematics, vol. 1, no. 3, pp. 257–275, 2003. [23] G. Anderson, A. Guionnet, and O. Zeitouni, An Introduction to Random Matrices, 1st ed. Cambridge: Cambridge University Press, 2009. [24] T. Tao, Topics in random matrix theory. Mathematical Society, 2012.
Graduate Studies in Mathematics, American
[25] G. Palla and G. Vattay, “Spectral transitions in networks,” New Journal of Physics, vol. 8, no. 12, p. 307, 2006. [26] J. N. Bandyopadhyay and S. Jalan, “Universality in complex networks: Random matrix analysis,” Physics Review E, vol. 76, p. 026109, Aug 2007. [27] S. Jalan and J. N. Bandyopadhyay, “Random matrix analysis of complex networks,” Physics Review E, vol. 76, p. 046107, Oct 2007. [28] I. J. Farkas, I. Der´enyi, A.-L. Barab´ asi, and T. Vicsek, “Spectra of real-world graphs: Beyond the semicircle law,” Physical Review E, vol. 64, no. 2, p. 026704, 2001. [29] D. Kim and B. Kahng, “Spectral densities of scale-free networks,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 17, no. 2, p. 026115, 2007. [30] S. N. Dorogovtsev, A. V. Goltsev, J. F. F. Mendes, and A. N. Samukhin, “Spectra of complex networks,” Physics Review E, vol. 68, p. 046109, Oct 2003. [31] K.-I. Goh, B. Kahng, and D. Kim, “Spectra and eigenvectors of scale-free networks,” Physics Review E, vol. 64, p. 051903, Oct 2001. [32] W. Aiello, F. Chung, and L. Lu, “A random graph model for power law graphs,” Experimental Mathematics, vol. 10, no. 1, pp. 53–66, 2001. [33] F. Chung and M. Radcliffe, “On the spectra of general random graphs,” The Electronic Journal of Combinatorics, vol. 18, no. 1, p. P215, 2011. [34] L. Lu and X. Peng, “Spectra of edge-independent random graphs,” The Electronic Journal of Combinatorics, vol. 20, no. 4, p. P27, 2013. [35] R. I. Oliveira, “Concentration of the adjacency matrix and of the Laplacian in random graphs with independent edges,” arXiv preprint arXiv:0911.0600, 2009. [36] V. M. Preciado, A. Jadbabaie, and G. Verghese, “Structural analysis of Laplacian spectral properties of large-scale networks,” IEEE Transactions on Automatic Control, vol. 58, no. 9, pp. 2338–2343, Sept 2013. [37] V. M. Preciado and A. Jadbabaie, “Moment-based spectral analysis of large-scale networks using local structural information,” IEEE/ACM Transactions on Networking (TON), vol. 21, no. 2, pp. 373–382, 2013. [38] V. M. Preciado and G. C. Verghese, “Low-order spectral analysis of the Kirchhoff matrix for a probabilistic graph with a prescribed expected degree sequence,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 56, no. 6, pp. 1231–1240, 2009. [39] V. M. Preciado and M. A. Rahimian, “Spectral moments of random matrices with a rank-one pattern of variances,” arXiv preprint arXiv:1409.5396, 2015. [40] P. Erd˝ os and A. R´enyi, “On random graphs. I,” Publ. Math. Debrecen, vol. 6, pp. 290–297, 1959.
18
V. M. Preciado and M. A. Rahimian
[41] ——, “On the evolution of random graphs,” Publ. Math. Inst. Hungar. Acad. Sci, vol. 5, pp. 17–61, 1960. [42] L. Erd˝ os, A. Knowles, H.-T. Yau, and J. Yin, “Spectral statistics of Erd˝ os–R´enyi graphs I: Local semicircle law,” The Annals of Probability, vol. 41, no. 3B, pp. 2279–2375, 2013. [43] ——, “Spectral statistics of Erd˝ os-R´enyi graphs II: Eigenvalue spacing and the extreme eigenvalues,” Communications in Mathematical Physics, vol. 314, no. 3, pp. 587–640, 2012. [44] Z. F¨ uredi and J. Koml´ os, “The eigenvalues of random symmetric matrices,” Combinatorica, vol. 1, pp. 233–241, 1981. [45] U. Feige and E. Ofek, “Spectral techniques applied to sparse random graphs,” Random Structures & Algorithms, vol. 27, no. 2, pp. 251–275, 2005. [46] E. Wigner, “Characteristic vectors of bordered matrices with infinite dimensions,” Annals of Mathematics, vol. 62, pp. 548–564, 1955. [47] ——, “On the distribution of the roots of certain symmetric matrices,” Annals of Mathematics, vol. 67, no. 2, pp. pp. 325–327, 1958. [48] P. Hagmann, M. Kurant, X. Gigandet, P. Thiran, V. J. Wedeen, R. Meuli, and J.-P. Thiran, “Mapping human whole-brain structural networks with diffusion MRI,” PLoS ONE, vol. 2, no. 7, p. e597, 2007. [49] P. Hagmann, L. Cammoun, X. Gigandet, R. Meuli, C. J. Honey, V. J. Wedeen, and O. Sporns, “Mapping the structural core of human cerebral cortex,” PLoS Biology, vol. 6, no. 7, p. e159, 07 2008. [50] S. S. Venkatesh, The Theory of Probability: Explorations and Applications. Cambridge University Press, 2012. [51] B. C. Arnold, N. Balakrishnan, and H. N. Nagaraja, A First Course in Order Statistics, John Wiley & Sons, Inc., New York, 1992. [52] W. Hoeffding, “Probability inequalities for sums of bounded random variables,” Journal of the American Statistical Association, vol. 58, no. 301, pp. 13–30, 1963. [53] P. Billingsley, Probability and Measure, 3rd ed. New York, NY: Wiley, 1995.
Moment-Based Spectral Analysis of Random Graphs with Given Expected Degrees
Victor M. Preciado†
&
M. Amin Rahimian
Department of Electrical and Systems Engineering, University of Pennsylvania (e-mail: [email protected])
In this paper, we analyze the limiting spectral distribution of the adjacency matrix of a random graph ensemble, proposed by Chung and Lu, in which a given expected degree sequence T
(n)
(n)
wn = (w1 , . . . , wn ) is prescribed on the ensemble. Let ai,j = 1 if there is an edge between the nodes {i, j} and zero other-
wise, and consider the normalized random adjacency matrix √ of the graph ensemble: An = [ai,j / n]ni,j=1 . The empirical spectral distribution of An denoted by Fn (·) is the empirical measure putting a mass 1/n at each of the n real eigenvalues of the symmetric matrix An . Under some technical conditions on the expected degrees sequence, we show that with probability one, Fn (·) converges weakly to a deterministic distribution F (·). Furthermore, we fully characterize this distribution by providing explicit expressions for the moments of F (·).
Keywords: Random Matrix Theory, Random Graph Models, Graph Spectrum
†
This work was supported by the National Science Foundation grants CNS-1302222 “NeTS: Medium: Collaborative Research: Optimal Communication for Faster Sensor Network Coordination”, and IIS1447470 “BIGDATA: Spectral Analysis and Control of Evolving Large Scale Networks”.
Spectral Analysis of Random Graphs
1
1. Introduction Understanding the relationship between structural and spectral properties of a network is a key question in the field of Network Science. Spectral graph methods (see [1, 2, 3], and references therein) have become a fundamental tool in the analysis of large complex networks, and related disciplines, with a broad range of applications in machine learning, data mining, web search and ranking, scientific computing, and computer vision. Studying the relationship between the structure of a graph and its eigenvalues is the central topic in the field of algebraic graph theory [4, 1, 2, 3, 5]. In particular, the eigenvalues of matrices representing the graph structure, such as the adjacency or the Laplacian matrices, have a direct connection to the behavior of several networked dynamical processes, such as spreading processes [6], synchronization of oscillators [7], random walks [8], consensus dynamics [9], and a wide variety of distributed algorithms [10]. The availability of massive databases describing a great variety of real-world networks allows researchers to explore their structural properties with great detail. Statistical analysis of empirical data has unveiled the existence of multiple common patterns in a large variety of network properties, such as power-law degree distributions [11], or the small-world phenomenon [12]. Random graphs models are the tool-of-choice to analyze the connection between structural and spectral network properties. Aiming to replicate empirical observations, a variety of synthetic network models has been proposed in the literature [11, 12]. The structural property that have (arguably) attracted the most attention is the degree distribution. Empirical studies show that the degree distribution of important real-world networks, such as the Internet [13], Facebook, or Twitter, are heavy-tailed and can be approximated using power-law distributions [14, 15]. We find in the literature that several random graphs models are able to model degree distributions from empirical data. One of best known models is the configuration model, originally proposed by Bender and Canfield in [16]. This model is able to fit a given degree sequence exactly (under certain technical conditions). Although many structural properties of this model have been studied in depth [17, 18], this model is not specially amenable to spectral analysis. A tractable alternative to the preferential attachment model was proposed by Chung and Lu in [19], and analyzed in [20, 21, 22]. In this model, which we refer to as the Chung-Lu model, an expected degrees sequence is prescribed onto a random graph ensemble, that can be algebraically described using a (random) adjacency matrix. Studies of the statistical properties of the eigenvalues of random graphs and networks are prevalent in many applied areas. Examples include the investigations of the spacing between nearest eigenvalues in random models [23, 24], as well as real-word networks [25, 26, 27]. Empirical observations highlight spectral features not observed in classical random matrix ensembles, such as a triangular like eigenvalue distribution in power-law networks [28, 29] or an exponential decay in the tails of the eigenvalue distribution [30, 31]. 1.1. Main Contributions In this work we offer an exact characterization for the eigenvalue spectrum of the adjacency matrix of the random graph models with a given expected degree sequence.
2
V. M. Preciado and M. A. Rahimian
This characterization is in terms of the moments of the eigenvalue distributions and it hinges upon application of the moments method for identifying the limiting spectral distribution of a related sequence of random matrices that is constructed from centralizing and normalizing the graph adjacencies. Accordingly, we give closed form expressions that describe the almost sure limits of the spectral moments and can be used to draw important conclusions about the graph spectrum and to bound several quantities of interest relating to the eigenvalue distribution of the graph adjacencies. The remainder of this paper is organized as follows. Preliminaries on the background and motivation of our study as well as the random graph model that we consider are presented in Section 2. Our main results on the asymptotic spectral moments of the adjacencies of random graphs and the characterization of the limiting spectral distributions for the normalized adjacencies are presented in Section 3, where we also include an outline of the proofs which are presented in detail in the Mathematical appendix at the end. In Section 4, we apply our results to a case where node degrees are obtained by random samples from the support of a preset function and show how our main results can be applied in analysis of the spectrum of large random graphs. Section 5 concludes the paper.
2. Background & Motivation 2.1. Chung-Lu Random Graph Model We consider the Chung-Lu random graph model introduced in [19] and analyzed in [20, 21, 22], in which an expected degree sequence given by the n non-negative entries of T (n) (n) the vector wn = (w1 , . . . , wn ) is prescribed over the nodes of the graph ensemble.1 In this model, each edge of the random (undirected) graph on n vertices (labeled by [n]) is realized independently of all other edges and in accordance with the probability measure P{·}. Let E {·} and Var {·} be the expectation and variance operators corresponding to P{·}. Following [32] we allow for self-loops. To each random graph we associate a (random) adjacency matrix, which is a zero-one matrix with the (i, j)-th entry being one if, and only if, there is an edge connecting nodes i and j. The number of edges incident to a vertex is the degree of that vertex, and by the volume of a graph we mean the sum of the degrees of its vertices. In this paper, our primary interest is in characterizing the asymptotic behavior of the eigenvalues of the random adjacency matrix as the graph size n increases. We consider the distribution of these eigenvalues over the real line and characterize this distribution through its moments sequence. Accordingly, the probability of there Pn (n) (n) (n) being an edge between nodes i and j is equal to ρn wi wj , where ρn = 1/ i=1 wi is the inverse expected volume. The adjacency relations in this random graph model are (n) √ represented by an n × n real-valued, symmetric random matrix An = [aij / n], where 1
Throughout this paper, R and N are the set of real and natural numbers, N0 = {0} ∪ N, n ∈ N is a parameter denoting the size of the random graph, [n] denotes {1, 2, . . . , n}, and card(X ) denotes the cardinality of set X . The n × n identity matrix is denoted by In , random variables are printed in boldface, matrices are denoted by capital letters. Every vector is marked by a bar over its lower case letter.
Spectral Analysis of Random Graphs
3
n o (n) (n) (n) (n) aij are independent zero-one random variables with E aij = ρn wi wj . As we shall √ see in the sequel, the 1/ n normalization is such that the distribution of the eigenvalues of the normalized adjacency An converges almost surely to a deterministic distribution that is uniquely characterized by its sequence of moments. As a main result of this paper, we explicate the technical conditions under which this convergence property holds true (cf. Assumptions 1 to 4 below). Furthermore, we proffer explicit expressions for calculating this moments sequence. These expressions, in turn allow us to upper or lower bound various quantities of interest pertaining to the spectrum of the random graph adjacencies (cf. Section 4 and discussions therein). Characterization of the convergence condition for the moments depends critically on the behavior of the extreme values in the expected degree sequence with the increasing size n. To that end, we consider two sequences {w ˆn : n ∈ N} and {w ˇn : n ∈ N} given (n) (n) by w ˆn = maxi∈[n] wi and w ˇn = mini∈[n] wi for all n ∈ N. Another quantity of interest whose evolution with the graph size n plays an important role in the preceding arguments is the second-order average degree d˜n given by [20], Pn (n) 2 n 2 X i=1 wi (n) = ρ wi = ρn kwn k22 . d˜n = Pn n (n) i=1 i=1 wi T
For our main results to hold true we need the expected degree sequence wn to satisfy the following conditions. Assumption Assumption Assumption Assumption
1 2 3 4
(Sparse and Graphical): ρn w ˆn2 < 1, ∀n and ρn w ˆn2 = o(1). (Logarithmic Growth): w ˆ n /w ˇn = O(log n). (Eigenvector Concentration): d˜n /ρn = o(n3 / log n). √ (Vanishing Effect of Centralization): d˜n = O( n log n).
2.2. Spectrum of the Chung-Lu Random Graph In 2003 by a series of results, Chung, Lu and Vu established important asymptotic properties of the spectra of the adjacency matrices of random graph models with given expected degree sequence [20, 21, 22]. A key result of theirs specifies the almost sure limit of the largest eigenvalue of the adjacency matrix as follows2 [20, Theorems 2.1 and 2.2]. √ Theorem 2.1 (Largest Eigenvalue of Random Graphs). If d˜n > w ˆn log n, then √ with probability one the largest eigenvalue of unnormalized adjacency, λn ( nAn ), is √ (1 + o(1))d˜n ; while if w ˆn > d˜n log2 n, then then the largest eigenvalue is almost surely √ (1 + o(1)) w ˆn .
2
Given three functions f (·), g(·), and h(·) we use the asymptotic notations f (n) = O(g(n)) and f (n) = o(h(n)) to signify the relations lim supn→∞ |f (n)/g(n)| < ∞ and limn→∞ |f (n)/h(n)| = 0, and we use f (n) (g(n)) to mean that f (n) = (1 + o(1))g(n).
4
V. M. Preciado and M. A. Rahimian
In particular, depending on the exponent of the power-law for a random graph model whose expected degree distribution obeys a power-law, either of the two regimes can be the case; and regardless, the largest eigenvalue is always with probability one less √ √ than 7 log n max{ w ˆn , d˜n }. In [20] similar conditions are established for the almost sure convergence of the k-th largest eigenvalues to the square root of the k-th largest expected degree, and [22] also considers relevant results concerning the spectra of other graph associated matrices such as the Laplacian. More recent results use the machinery of concentration inequalities to investigate the behavior of the graph spectra for random graphs with independent edges [33, 34, 35]. In [35] the author shows concentration of the spectral radius norm for the Laplacian and Adjacency matrices around their expectations. These results are improved by Chung and Radcliffe [33] who use a Chernoff type inequality and approximate the eigenvalues by those of the expected matrices. The √ ˆn log n) and it approximation bound in [33] for the eigenvalues of the adjacency is O( w √ ˆn by [34]. is later improved to (2 + o(1)) w 3. Main Result Here, we offer a moment-based characterization of the limiting distribution of the eigenvalues of the Chung-Lu random graph model given Assumptions 1 to 4 on the expected degree sequence. The moments sequence provides a versatile tool in the spectral analysis √ of complex networks [36, 37, 38]. It is worth highlighting that in the 1/ n normalization regime that we work, as n → ∞ the largest eigenvalue escapes to infinity. This is a common case when investigating the limiting distribution of a sequence of distributions that some mass escapes to infinity, which can cause the limit distribution to be not a probability distribution (does not integrate to one), in which case the underlying sequence of distribution is not “tight”. However, as we show in this paper, the tightness property holds for the sequence of spectral distributions in the Chung-Lu random graph model. It is because the mass that is escaping to infinity (a single largest eigenvalue) is asymptotically vanishing itself: as n → ∞, the contribution of a single eigenvalue to the continuous √ limiting spectral distribution in the 1/ n normalization regime is vanishingly small. By the same token, our results complement the characterization of the largest eigenvalue √ by [20, 21, 22]; which focuses on the adjacency matrix itself, as opposed to the (1/ n)normalized version, and investigates the growth rate and concentration of the largest eigenvalue as described in Subsection 2.2. To describe our main results we need to introduce some terminology. Let λ1 (An ) ≤ λ2 (An ) ≤ . . . ≤ λn (An ) be n real-valued random variables representing the n eigenvalues of the random matrix An ordered from the smallest to the largest. We define Ln {·} = Pn (1/n) i=1 δλi (An ) {·}, as the random probability measure on the real line that assigns mass uniformly to each of the n eigenvalues of the random matrix An . Here, δx {·} is the probability measure on R assigning unit mass to point x ∈ R and zero elsewhere. The corresponding distribution Fn (x) = Ln {(−∞, x]} = (1/n)card({i ∈ [n] : λi (An ) ≤ x}), is a random variable for each x ∈ R and is referred to as the empirical spectral distri(n) bution for the random matrix An . Moreover, we define mk = (1/n)trace(Akn ) R +∞ (ESD) k = −∞ x d Fn (x) as the real-valued random variable which is the k-th spectral moment
Spectral Analysis of Random Graphs
5
of the random matrix An . Our main results constitute in concluding the almost sure and weak convergence of the empirical spectral distribution of the adjacency An to a deterministic distribution F (·). We call this distribution the limitingR spectral distribution +∞ (LSD) and we characterize it through its moments sequence, mk = −∞ xk dF (x). These moments are derived in terms of the limiting average power sums of the normalized degree sequence defined as follows. √ (n) For each i ∈ N, we define the limiting normalized degrees σi = limn→∞ ρn wi . The limiting average of the k-th power sum of the sequence {σi , i ∈ N} is given by Λk = Pn limn→∞ (1/n) i=1 σik . Note from Assumption 1 that σi < 1 for all 1, which in turn implies that Λk < 1 for all k. According to our main result, the spectral moments of the limiting spectral distribution F (·) are specified as follows.
Theorem 3.1 (Spectral Moments of Random Graphs). Consider the random graph model with given expected degree sequence wn , satisfying Assumptions 1 to 4. With probability one, Fn (·) converges weakly to F (·) and F (·) is the unique distribution function R +∞ satisfying ∀k ∈ N, −∞ xk dF (x) = mk where for all s ∈ N0 , m2s+1 = 0, and X 2 s+1 (3.1) Λr1 Λr2 ...Λrss , m2s = s + 1 r1 , ..., rs 1 2 r∈Rs
with r = (r1 , r2 , ..., rs ), Rs = {r ∈ Ns0 :
Ps
j=1 rj
= s + 1,
Ps
j=1
j rj = 2s}.
3.1. Proof Outline The crux of the argument is in showing that for each k ∈ N, the k-th spectral moments (n) mk converge almost surely to mk , thence concluding by the method of moments that with probability one the empirical spectral distributions Fn (·) converge weakly to F (·). A centralization idea would allow us to proceed in parallel with the proof steps in our earlier results in [39], concerning random matrices with independent zero-mean entries and rank-one pattern of variances. To begin, we consider the centralized version of the adjacency An given by i h √ (n) ˆ n = An − E {An } = (1/ n)ˆ aij . (3.2) A (n)
Note that the entries ˆ aij have zero means and asymptotically rank one pattern of variances given by o 2 n (n) (n) (n) (n) (n) (n) Var ˆ aij = E ˆ aij = ρn wi wj (1 − ρn wi wj ) σi σj , (3.3) (n)
(n)
(n)
since by Assumption 1, ρn wi wj = o(1). Furthermore, ˆ aij entries are all uniformly n o (n) almost surely bounded: P |ˆ aij | ≤ 1 = 1. Indeed, except for the fact that the pattern ˆ n is asymptotically rank one rather than exactly rank of variances for the entries of A ˆ one, the random matrix An together with the sequence {σi , i ∈ N}, satisfy the all four necessary conditions of the rank one ensemble investigated in [39]. It is therefore to be
6
V. M. Preciado and M. A. Rahimian
expected that save for a few minor refinements, the moment-based analysis of the random ˆ n just as well. matrix ensemble in [39] applies to A To proceed we introduce some necessary notations. Similarly to An , we consider the ˆ n ordered from the smallest to the largest as λ1 (A ˆ n ) ≤ λ 2 (A ˆ n) ≤ . . . ≤ eigenvalues of A (n) k ˆ ˆ λn (An ) and define the random variable m ˆ k = (1/n)trace(An ) to be its k-th spectral (n) (n) moment. Also let m ¯ k = E{m ˆ k } be the expected spectral moments for all k, n. The proof of our main result proceeds as follows. (n) (n) Lemma A1.2 of the Appendix ensures that under Assumptions 3 and 4, m ˆ k mk , almost surely for each k ∈ N. Therefore, the effect of centralization on the spectral moments is asymptotically vanishing. Next Lemma A2.2 provides the asymptoticly exact (n) expressions: m ¯ k mk for the expected spectral moments under Assumptions 1 and 2. Lemma A2.1 implies that the same expressions in fact describe the almost sure limits (n) of the spectral moments: m ˆ k mk for all k ∈ N. The latter, together with the earlier (n) results, imply that under Assumptions 1 to 4: mk mk , almost surely for all k ∈ N. This pointwise almost sure convergence of the moments sequence would in turn imply the weak convergence of the empirical distributions Fn (·) to the deterministic distribution F (·) characterized uniquely by the sequence of limiting spectral moments. This fact is established in Lemma A3.1, completing the proof. 3.2. The Case of Erd˝ os-R´ enyi Random Graphs In Erd˝os-R´enyi random graphs denominated popularly as Gn,p each edge is realized with a probability p and independently of all else. This is a special case of the random graph model with given expected degrees, where the expected degrees are fixed and the expected degree sequence wn is equal to (np, np, . . . , np)T . Ever since its introduction in the late 1950s by Erd˝os and R´enyi [40, 41], properties of this well-known class of random graphs have been extensively studied and they continue to attract attention [42, 43]. Indeed, the seminal work of F¨ uredi and Koml´ os [44] immediately gives important asymptotic properties of the spectra of Erd˝ os-R´enyi Random Graphs; with probability one, putting its largest eigenvalue at (1 + o(1))np and upper-bounding the absolute values of the rest p by (2 + o(1)) np(1 − p). More recently, Feige and Ofek [45] have shown that under mild √ conditions on p, the largest eigenvalue of adjacency is almost surely pn + O( pn) and √ all other eigenvalues are almost surely O( pn). Fig. 1(a) depicts the histogram of eigenvalues for a particular realization of the un√ normalized adjacency nAn , with n = 1000 nodes and p = 0.05. In particular, we can √ observe that the largest eigenvalue λ1 ( nAn ) is located away from the remaining smaller ˆ n defined in (3.2). We plot a ones. Let us also consider the centralized adjacency matrix A typical realization of the eigenvalue histogram of the unnormalized centralized adjacency √ ˆ nAn in Fig. 1(b). Notice that, as pointed out in Lemma A1.2, the effect of centralizing the adjacency matrix An (w) is to cancel out the largest eigenvalue, pushing it back to zero while the bulk of eigenvalues remains (almost) unperturbed. Subsequently, the √ effect of normalization on the limiting spectral distribution in the 1/ n-normalization regime is vanishing. This can be also noticed in Fig. 1(c), where we plot the empirical ˆ n (x). Indeed under the 1/√n-normalization that we spectral distributions Fn (x) and F
Spectral Analysis of Random Graphs
7
√ investigate the limiting spectral distributions, the largest eigenvalue of An grows as np, √ escaping almost surely to infinity; every other eigenvalue is almost surely O( p), being asymptotically compactly supported.3
Figure 1. (a) Eigenvalue histogram of the Erd˝ os-R´ enyi random graph; (b) eigenvalue histogram of the centralized adjacency; (c) spectral distributions of the normalized adjacency An (blue solid) and the ˆ n (red dashed). centralized version A
4. Degrees Specified by Random Uniform Sampling (n)
Here we consider the case where the expected degree of each node i is specified as wi = fn (xi ), where fn (·) are given functions with a common support normalized to be the unit interval [0, 1], and {xi , i ∈ N} is a random sample, uniformly and independently drawn from the unit interval. To illustrate our results, let us assume that fn (x) = ∆n e−αn x , 0 ≤ x ≤ 1, and consider a Chung-Lu random graph model with the expected degree (n) sequence specified as wi = ∆n e−αn xi , i ∈ [n]. Exponential degree distributions have been observed in practice, and are known to be good descriptors of the structural brain network data that are obtained through diffusion imaging techniques [48, 49]. The almost sure asymptotic expression for the second-order average degree d˜n can be obtained from a Monte Carlo average [50, Section XVI.3], and is given by the strong law of large numbers as follows, Pn (n) 2 R1 2 R 1 2 −2α x n f (x)dx ∆ e dx i=1 wi ∆n (1 − e−2αn ) 0 n ˜ R1 = R01 n = , (4.1) dn = Pn (n) 2(1 − e−αn ) f (x)dx ∆n e−αn x dx i=1 wi 0 n 0 √ ˜ n > ∆n log n, and the and we know from Theorem 2.1 that if ∆n > log2 n, then d √ ˜ n . We now use Theorem largest eigenvalue is almost surely given by λ1 ( nAn ) d 3.1 to write closed-form expressions for the asymptotic spectral moments of An , the normalized adjacency of the random graph whose expected degree sequence is given by 3
It is possible to derive the limiting spectral distribution of this random graph ensemble from Theorem √ 3.1 under Assumptions 1 to 4, provided that p = pn = O(log n/ n). Then w ˆn = w ˇn = npn , ρn = −1 n−2 pn , and d˜n = npn all satisfy the requirements of Assumptions 1 to 4 and Theorem 3.1 gives the s pn 2s , which correspond to a semi circular distribution the asymptotic spectral moments as m2s = s+1 s √ √ supported over [−2 pn , +2 pn ], [46, 47]. Applying the results of F¨ uredi and Koml´ os [44] for the limiting spectral distribution of random matrices with independent zero-mean entries and identical variances provides that the same semi-circular distribution still describes the LSD of centralized and normalized adjacency matrix even when pn = p is fixed and is not decreasing with n.
8
V. M. Preciado and M. A. Rahimian (n)
(n)
wTn = (w1 , . . . , wn ). We begin by calculating the almost sure asymptotic expression of the inverse expected volume as follows, Z 1 Z 1 n X n∆n 1 (n) ∆n e−αn x dx = = fn (x)dx = n 1 − e−αn . (4.2) wi n ρ n i=1 αn 0 0
We now proceed to verify the qualifying conditions for applying Theorem 3.1 to this random graph model. First note the following almost sure asymptotic identities for the maximum and minimum degrees, (n)
∆n ,
(n)
∆n e−αn ,
w ˆ n = max wi i∈[n]
ˇ n = min wi w i∈[n]
(4.3)
both of which can be verified easily from the corresponding order statistics for the uniform √ distribution on the unit interval.4 If we set ∆n = n log n and αn = log log n, then from the set of almost sure asymptotic identities in (4.1), (4.2) and (4.3) we get √ √ ˜ n ∆n /2 = n log n/2 = O( n log n), d w ˆn eαn = log n = O(log n), ˇn w r √ ∆n α n log n log log n √ ρn w ˆn = = o(1), n n1/4 3 ˜n d n∆2n n n2 log2 n =o , = ρn 2αn 2 log log n log n Hence, Assumptions 1 to 4 are all satisfied and we can apply Theorem 3.1 to obtain the closed-form expressions for the limiting spectral moments of the normalized adjacency An . First note that the k-th order limiting averages Λk are almost surely given by Z 1 Z 1 √ k k ( ρn ∆n )k √ √ ρn fn (x) dx = ρn ∆n e−αn x dx = 1 − e−kαn . Λk kαn 0 0
4
More specifically, if we note that mini∈[n] xi and maxi∈[n] xi are respectively distributed as Beta(1, n) and Beta(n, 1) variables [51, Chapter 2], then the claimed almost sure limits follow by the Borel-Cantelli lemma. To see how, consider their expected values: E{mini∈[n] xi } = 1/(n + 1) and E{maxi∈[n] xi } = n/(n + 1), and apply the Chebyshev inequality to their quadratically decaying common variance, Var{maxi∈[n] xi } = Var{mini∈[n] xi } = n/ (n + 1)2 (n + 2) .
Spectral Analysis of Random Graphs
9
Under these conditions, we have the following closed-form expression for the asymptotic spectral moments of An : √ s s X 2 s+1 Y X 2 s+1 Y ( ρn ∆n )krk ρsn ∆2s n m2s = k −rk s+1 s + 1 r1 , ..., rs (kαn )rk s + 1 r1 , ..., rs αn r∈Rs
=
r∈Rs
k=1
k=1
s X 2 s+1 Y logs n k −rk , ns/2 log log n r∈R s + 1 r1 , ..., rs k=1
(4.4)
s
Ps Ps where in the second equality we have used the identities k=1 rk = s + 1 and k=1 krk = 2s. The histogram of the eigenvalues for the normalized adjacency matrix An of a particular realization with parameters ∆ = 10, α = 1, and n = 1000 is plotted in Fig. ˆ n = An − [√ρn w(n) w(n) ]n 2. The largest eigenvalue of the centralized adjacency A i,j=1 in i j ˆ n ) = 5.6214. We can upper and lower bound the largest this realization is given as λn (A eigenvalue using the k-th spectral moments as follows [24, Equation (2.66)]: 1/k ˆ kn )1/k ≤ λn (A ˆ n ) ≤ n · trace(A ˆ kn ) trace(A . (4.5)
If we consider k = 20 in (4.5) and use the asymptotic spectral moment m20 available ˆ ˆ 20 from (4.4) to replace for trace(A n ), then the lower and upper bounds on λn (An ) are as 1/20 1/20 follows: (m20 ) = 4.3193 and (n · m20 ) = 6.1011. These values compare reasonably ˆ with the empirically observed value λn (An ) = 5.6214. Using the techniques in [37], we can formulate semi-definite programs that improve the bounds in in (4.5) by taking into account the knowledge of all spectral moments up to a fixed order, as described in [39, Section 4].
60 50 40 30 20 10 0 Figure 2.
-6
-4
-2
0
2
4
6
Eigenvalue histogram of a sample realization from the random graph model with exponential degree sequence and n = 1000 vertices.
10
V. M. Preciado and M. A. Rahimian 5. Conclusions
In this work, we investigated the asymptotic behavior of the eigenvalues of the adjacency of random graph models with specified expected degrees sequence. We showed that in √ the 1/ n normalization regime that we consider, and under some technical assumptions on the expected degrees sequence, the empirical spectral distribution of the adjacency converges weakly to a deterministic distribution, and we gave closed-form expressions for its limiting spectral moments. We illustrated the application of our results to spectral analysis of large-scale networks with exponential degree distributions. The latter are found to be good descriptors for the structural brain network data, obtained through diffusion imaging. Furthermore, closed-form expressions of the moments allow us to bound various quantities of interest relating to the spectrum such as the spectral radius. These bound can be strengthened using SDP formulations that take into account the knowledge of all spectral moments up to a fixed order.
Spectral Analysis of Random Graphs
11
Appendix: Proof of the Main Result The argument leading to the proof of our main convergence result in Theorem 3.1 is executed in three steps. We begin by showing in Section A1 that with probability (n) (n) ˆn one limn→∞ mk = limn→∞ m ˆ k , i.e. the adjacency An and its centralized version A share the same almost sure limits for their spectral moments. Next in Section A2 we (n) refine our arguments from [39] leading to the almost sure limits limn→∞ m ˆ k = mk , with mk given by (3.1) as in Theorem 3.1. Finally in Section A3 we apply the method of moments to conclude the weak and almost sure convergence of the empirical spectral distributions Fn (·) to the deterministic distribution F (·) determined by the moments sequence mk , k ∈ N that we have obtained earlier. A1. Effect of Centralization on the Spectral Moments The following set of results measures the effect of the centralization in (3.2) by comparing ˆ n and An asymptotically as n → ∞. Indeed, centralization the spectral moments of A by subtracting the mean E {An } from the adjacency An as in (3.2), has the effect of shifting the largest eigenvalue back to zero and symmetrizing the spectrum. Example 1 demonstrates this shifting; however, in the sequel we shall show that the subsequent effect on the spectral moments is asymptotically vanishing provided that certain mild assumptions on the degree sequences are satisfied. √ ˆn log n, then λ1 (An ) d˜n with probability We know from Theorem 2.1 that if d˜n > w one. We use a variation of this result to prove that the vector of expected degrees is asymptotically almost surely an eigenvector of An associated its largest eigenvalue. In particular, as λn (An ) concentrates around d˜n , the vector (1/n)An wn also concentrates 1 ˜ d w . This is important when characterizing the effect of the round the vector n√ n n n centralization in (3.2) in light of the fact that T T ρn ρn d˜n E {An } = √ wi wj = √ wn wn = √ wn wn . (A1.1) 2 n n nkwn k2 i,j∈[n] Lemma A1.1 (Eigenvector Concentration). Under Assumption 3, it is true that 1 1 ˜ √ n An w n n n dn w n almost surely. Proof. The i-th component of An wn is a random variable given by: (n) n X aij (n) √ [An wn ]i = w . n j j=1 (n)
(n)
(A1.2) (n)
(n)
Since aij is a Bernoulli random variable with P{aij = 1} = ρn wi wj , so that we have 2 d˜n (n) (n) ρn w j = √ wi , n j=1 n
(n)
w E {[An wn ]i } = √i
n X
12
V. M. Preciado and M. A. Rahimian
√ (n) (n) √ and E {An wn } = (d˜n / n)wn . Next note that each of the summands aij wj / n in (n) (n) √ (n) √ (A1.2) are independent bounded random variables satisfying aij wj / n ∈ [0, wj / n] almost surely. Hence, we can apply Hoeffding’s inequality [52, Theorem 2 ] to get that for each i and any ε > 0 ( ) 3 2 ˜ dn (n) 1 −2n ε −2n3 ρn ε2 /d˜n . P [An wn ]i − √ wi ≥ ε ≤ 2 exp P 2 = 2e n n (n) n w i i=1
Next note that given d˜n /ρn = o(n3 / log n) per Assumption 3, with ε as in above and for any α > 1, we get that when n is large enough, d˜n /ρn < 2n3 ε2 /(α log n); whence 3 2 ˜ 2e−2n ρn ε /dn < 2/nα forms a summable series in n, and by the Borel-Cantelli lemma [53, Theorem 4.3] we get that ( ) ˜n (n) 1 d P [An wn ]i − √ wi ≥ ε, i.o. = 0, n n n which holds true for any ε > 0, and therefore we have ) ( d˜n (n) 1 = 1. P lim [An wn ]i = lim √ wi n→∞ n n n→∞ n
The claimed concentration of eigenvector around wn now follows by the countable intersections of the above almost sure events over all i ∈ N. We can now proceed to give conditions under which the spectral moments of An and ˆ n are asymptotically almost surely identical, and therefore the effect of centralization A on spectral moments is asymptotically vanishing.
Lemma A1.2 (Vanishing Effect of Centralization). Under Assumptions 3 and 4, (n) (n) m ˆ k mk , almost surely for each k ∈ N. Proof. To begin consider the k = 1 case. From (3.2) we have (n)
=
m ˆ1
1 ˆ n ) = 1 trace(An − E{An }). trace(A n n
From (A1.1) we know that T d˜n d˜n trace(wn wn ) = √ , 2 nkwn k2 n √ ˆ n ) = trace(An ) − d˜n / n, is true for all n, and in wherefrom it follows that trace(A particular with probability one as n → ∞. For general k ∈ N, we have
trace(E{An }) = √
(n)
m ˆk
=
1 ˆ kn ) = 1 trace(An − E{An })k . trace(A n n
Spectral Analysis of Random Graphs
13
To proceed, consider the binomial expansion of (An − E{An })k consisting of a sum of the product of non-commutative elements as follows: k−2 Akn + Ak−1 n (−E{An }) + An (−E{An })An + . . . 2 k−3 + Ak−2 n (−E{An }) + An (−E{An })An (−E{An }) + . . .
Consider any product term of the form, Π(k) = Akn1 (−E{An })k2 Akn3 . . . (−E{An })kp ,
(A1.3)
where k = (k1 , . . . , kp ) is an integer partition of k consisting of p > 1 positive integers satisfying k1 + . . . + kp−1 = k. Let k˜ be the sum of evenly indexed integers: k˜ = k2 + k4 + . . . + k2·bp/2c . We claim that as n → ∞, d˜kn 1 ˜ trace(Π(k)) → (k/2)+1 (−1)k , n n
(A1.4) 2(ki −1)
T
almost surely. To see why, first note that for any ki , (wn wn )ki = kwn k2 taking the ki -th power of both sides in (A1.1), we get (−E{An })ki =
T
wn wn , so
T (−d˜n )ki wn wn , 2 k /2 i n kwn k2
and replacing in (A1.3) yields Π(k) =
(−d˜n ) ˜ nk/2
˜ k
T
Akn1 (
T
wn wn wn wn )Ak3 . . . ( ). kwn k22 n kwn k22
(A1.5)
Under Assumption 3, from Lemma A1.1 we know that (1/n)An wn (d˜n /n3/2 )wn almost surely. Multiplying both sides by Akni −1 for any integer ki yields 1 ki d˜kni An wn (k /2)+1 wn , n n i almost surely. Hence, taking the limits of both sides in (A1.5) we get T 1 d˜nk wn wn ˜ Π(k) (−1)k (k/2)+1 ( ), n kwn k22 n T
almost surely. Taking the trace of both sides and using the fact that trace(wn wn /kwn k22 ) = 1, leads to (A1.4) as claimed. The above applies invariably to any of the product terms appearing in the binomial expansion of (An − E{An })k , with the exception of the leading term, trace(Akn ), which does not simplify any further. Next note that given any 1 ≤ k˜ ≤ k, the number of terms Π(k) in the binomial expansion for which (A1.4) holds true, is exactly kk˜ ; wherefore we get that with probability one, k ˜k X 1 1 d k ˜ ˆ kn ) = trace(An − E{An })k trace(Akn ) + n trace(A (−1)k ˜ . n n k nk/2 ˜ k=1
14
V. M. Preciado and M. A. Rahimian
We next use a trite identity, 0 = (1 − 1)k = 1 + 1 ˆ kn ) 1 trace(A n n
Pk
(−1) ˜ k=1
trace(Akn ) −
almost surely. The claim now follows upon noting that √ is true because d˜n = O( n log n) per Assumption 4.
˜ k k ˜ k
d˜nk nk/2
!
to get ,
1 ˜ √ k n (dn / n)
= o(1), ∀k ∈ N, which
A2. Almost Sure Limits of the Spectral Moments In this section we shall establish that the asymptotic relations for the almost sure limits ˆ n are indeed given by (3.1) in of the spectral moments of the centralized adjacency A (n) Theorem 3.1, i.e. we have m ˆ k mk , almost surely for each k ∈ N. The underlying arguments parallel those in [39] and some details are omitted here in the interest of expositional brevity and with no peril to the formality of the arguments, as the omitted details are replicas of those in [39]. In particular, Lemma A3.13 in [39] on the almost sure convergence of the spectral moments holds true in the case of our random matrix ˆ n and with no change to the proof steps therein. Accordingly, we have: ensemble A Lemma A2.1 (Almost Sure Convergence of the Spectral Moments). For any (n) (n) k ∈ N, if m ¯ k mk , then m ˆ k mk almost surely. ˆ n it Hence, for investigating the almost sure convergence of the spectral moments of A suffices to consider their expectations in the limit. This is achieved by first identifying the terms that asymptotically dominate the behavior of each moment of any order as in Subsection 3.1 of [39], and then deriving the asymptotically exact expressions for each of the identified terms following [39, Subsection 3.2]. The analysis leading to Theorem ˆ n , provided that we 3.4 and Theorem 3.5 of [39] hold true verbatim in the case of A √ √ ˆn and ρn w ˇn respectively. By the same token, replace σ ˆn and σ ˇn therein, by ρn w ˆ n , establishing Theorem 3.6 in [39] can be recycled for our random matrix ensemble A (3.1) as the asymptotically exact expressions for the even moments. Here, in addition √ (n) to replacing σ ˆn and σ ˇn , we should also change every σi term in that proof to ρn wi for finite n; however, this adjustment does not influence the conclusion of the theorem which concerns the limit as n → ∞ of the dominant term identified by Theorem 3.4, since 2 √ (n) (n) we have defined σi = limn→∞ ρn wi , ∀i and per (3.3), E ˆ aij σi σj . Thereby, combining the results of Theorem 3.4, Theorem 3.5 and 3.6 in [39] and after the minor adjustments indicated above, we obtain: (n)
Lemma A2.2 (Limiting Spectral Moments). Under Assumptions 1 and 2, m ¯ 2s (n) m2s and m ¯ 2s−1 = o(1), for all s ∈ N.
Spectral Analysis of Random Graphs
15
A3. Weak Convergence of the Spectral Distribution The results of Sections A1 and A2 thus far enable us to claim that under Assumptions (n) 1 to 4, the spectral moments sequence {mk , k ∈ N} converges pointwise almost surely to the deterministic sequence {mk , k ∈ N}. Coup de grˆ ace is to conclude the almost sure and weak convergence of the empirical spectral distributions from the pointwise almost sure convergence of their moments sequence; it is the province of the method of moments whose details are spelled out in [39, Appendix A2] and they can be reused word for word in the case of our empirical spectral distributions Fn (·): (n)
Lemma A3.1 (Method of Moments, Theorem 3.1 of [39]). If mk mk almost surely for all k ∈ N, then with probability one Fn (·) converges weakly R +∞to F (·) as n → ∞, where F (·) is the unique distribution function satisfying ∀k ∈ N, −∞ xk dF (x) = mk .
In fact, Theorem 3.1 of [39] provides more and it is further true that Fn (·) converges weakly, almost surely, to F (·) as n → ∞ (cf. [39, Appendix A1] for the definitions of the two modes of weak convergence). We have thus pieced together all the ingredients required of our main result characterizing the moment sequence of the almost sure weak √ limit of the (1/ n)-normalized adjacency matrix of random graph models with specified expected degree sequence.
16
V. M. Preciado and M. A. Rahimian
References
[1] F. Chung, Spectral Graph Theory (CBMS Regional Conference Series in Mathematics, No. 92). American Mathematical Society, Dec. 1996. [2] D. M. Cvetkovic, M. Doob, and H. Sachs, Spectra of graphs: Theory and application. Academic Press, New York, 1980, vol. 413. [3] D. Cvetkovi´c, P. Rowlinson, and S. Simi´c, “An introduction to the theory of graph spectra,” Cambridge-New York, 2010. [4] N. Biggs, Algebraic Graph Theory, 2nd ed. Cambridge: Cambridge University Press, 1993. [5] R. Merris, “Laplacian matrices of graphs: a survey,” Linear Algebra and its Applications, vol. 197-198, no. 0, pp. 143 – 176, 1994. [6] P. Van Mieghem, J. Omic, and R. Kooij, “Virus spread in networks,” IEEE/ACM Transactions on Networking, vol. 17, no. 1, pp. 1–14, Feb 2009. [7] L. M. Pecora and T. L. Carroll, “Master stability functions for synchronized coupled systems,” Physics Review Letters, vol. 80, no. 10, pp. 2109–2112, March 1998. [8] L. Lov´ asz, “Random walks on graphs: A survey,” Combinatorics, vol. 2, no. 1, pp. 1–46, 1993. [9] R. Olfati-Saber, “Flocking for multi-agent dynamic systems: Algorithms and theory,” IEEE Transactions on Automatic Control, vol. 51, no. 3, pp. 401–420, 2006. [10] N. A. Lynch, Distributed algorithms. Morgan Kaufmann, 1996. [11] A. Barab´ asi and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, no. 5439, pp. 509–512, 1999. [12] D. Watts and S. Strogatz, “Collective dynamics of small-world networks,” Nature, no. 393, pp. 440–442, 1998. [13] M. Faloutsos, P. Faloutsos, and C. Faloutsos, “On power-law relationships of the Internet topology,” in SIGCOMM, 1999, pp. 251–262. [14] A. Clauset, C. R. Shalizi, and M. E. J. Newman, “Power-law distributions in empirical data,” SIAM Review, vol. 51, no. 4, pp. 661–703, 2009. [15] A.-L. Barab´ asi and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, no. 5439, pp. 509–512, 1999. [16] E. A. Bender and E. R. Canfield, “The asymptotic number of labeled graphs with given degree sequences,” Journal of Combinatorial Theory, Series A, vol. 24, no. 3, pp. 296–307, 1978. [17] M. Molloy and B. Reed, “The size of the giant component of a random graph with a given degree sequence,” Combinatorics, Probability and Computing, vol. 7, no. 03, pp. 295–305, 1998. [18] S. Chatterjee, P. Diaconis, and A. Sly, “Random graphs with a given degree sequence,” The Annals of Applied Probability, vol. 21, no. 4, pp. 1400–1435, 2011. [19] F. Chung and L. Lu, “Connected components in random graphs with given expected degree sequences,” Annals of combinatorics, vol. 6, no. 2, pp. 125–145, 2002. [20] F. Chung, L. Lu, and V. Vu, “Eigenvalues of random power law graphs,” Annals of Combinatorics, vol. 7, no. 1, pp. 21–33, 2003.
Spectral Analysis of Random Graphs
17
[21] ——, “Spectra of random graphs with given expected degrees,” Proceedings of the National Academy of Sciences, vol. 100, no. 11, pp. 6313–6318, 2003. [22] ——, “The spectra of random graphs with given expected degrees,” Internet Mathematics, vol. 1, no. 3, pp. 257–275, 2003. [23] G. Anderson, A. Guionnet, and O. Zeitouni, An Introduction to Random Matrices, 1st ed. Cambridge: Cambridge University Press, 2009. [24] T. Tao, Topics in random matrix theory. Mathematical Society, 2012.
Graduate Studies in Mathematics, American
[25] G. Palla and G. Vattay, “Spectral transitions in networks,” New Journal of Physics, vol. 8, no. 12, p. 307, 2006. [26] J. N. Bandyopadhyay and S. Jalan, “Universality in complex networks: Random matrix analysis,” Physics Review E, vol. 76, p. 026109, Aug 2007. [27] S. Jalan and J. N. Bandyopadhyay, “Random matrix analysis of complex networks,” Physics Review E, vol. 76, p. 046107, Oct 2007. [28] I. J. Farkas, I. Der´enyi, A.-L. Barab´ asi, and T. Vicsek, “Spectra of real-world graphs: Beyond the semicircle law,” Physical Review E, vol. 64, no. 2, p. 026704, 2001. [29] D. Kim and B. Kahng, “Spectral densities of scale-free networks,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 17, no. 2, p. 026115, 2007. [30] S. N. Dorogovtsev, A. V. Goltsev, J. F. F. Mendes, and A. N. Samukhin, “Spectra of complex networks,” Physics Review E, vol. 68, p. 046109, Oct 2003. [31] K.-I. Goh, B. Kahng, and D. Kim, “Spectra and eigenvectors of scale-free networks,” Physics Review E, vol. 64, p. 051903, Oct 2001. [32] W. Aiello, F. Chung, and L. Lu, “A random graph model for power law graphs,” Experimental Mathematics, vol. 10, no. 1, pp. 53–66, 2001. [33] F. Chung and M. Radcliffe, “On the spectra of general random graphs,” The Electronic Journal of Combinatorics, vol. 18, no. 1, p. P215, 2011. [34] L. Lu and X. Peng, “Spectra of edge-independent random graphs,” The Electronic Journal of Combinatorics, vol. 20, no. 4, p. P27, 2013. [35] R. I. Oliveira, “Concentration of the adjacency matrix and of the Laplacian in random graphs with independent edges,” arXiv preprint arXiv:0911.0600, 2009. [36] V. M. Preciado, A. Jadbabaie, and G. Verghese, “Structural analysis of Laplacian spectral properties of large-scale networks,” IEEE Transactions on Automatic Control, vol. 58, no. 9, pp. 2338–2343, Sept 2013. [37] V. M. Preciado and A. Jadbabaie, “Moment-based spectral analysis of large-scale networks using local structural information,” IEEE/ACM Transactions on Networking (TON), vol. 21, no. 2, pp. 373–382, 2013. [38] V. M. Preciado and G. C. Verghese, “Low-order spectral analysis of the Kirchhoff matrix for a probabilistic graph with a prescribed expected degree sequence,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 56, no. 6, pp. 1231–1240, 2009. [39] V. M. Preciado and M. A. Rahimian, “Spectral moments of random matrices with a rank-one pattern of variances,” arXiv preprint arXiv:1409.5396, 2015. [40] P. Erd˝ os and A. R´enyi, “On random graphs. I,” Publ. Math. Debrecen, vol. 6, pp. 290–297, 1959.
18
V. M. Preciado and M. A. Rahimian
[41] ——, “On the evolution of random graphs,” Publ. Math. Inst. Hungar. Acad. Sci, vol. 5, pp. 17–61, 1960. [42] L. Erd˝ os, A. Knowles, H.-T. Yau, and J. Yin, “Spectral statistics of Erd˝ os–R´enyi graphs I: Local semicircle law,” The Annals of Probability, vol. 41, no. 3B, pp. 2279–2375, 2013. [43] ——, “Spectral statistics of Erd˝ os-R´enyi graphs II: Eigenvalue spacing and the extreme eigenvalues,” Communications in Mathematical Physics, vol. 314, no. 3, pp. 587–640, 2012. [44] Z. F¨ uredi and J. Koml´ os, “The eigenvalues of random symmetric matrices,” Combinatorica, vol. 1, pp. 233–241, 1981. [45] U. Feige and E. Ofek, “Spectral techniques applied to sparse random graphs,” Random Structures & Algorithms, vol. 27, no. 2, pp. 251–275, 2005. [46] E. Wigner, “Characteristic vectors of bordered matrices with infinite dimensions,” Annals of Mathematics, vol. 62, pp. 548–564, 1955. [47] ——, “On the distribution of the roots of certain symmetric matrices,” Annals of Mathematics, vol. 67, no. 2, pp. pp. 325–327, 1958. [48] P. Hagmann, M. Kurant, X. Gigandet, P. Thiran, V. J. Wedeen, R. Meuli, and J.-P. Thiran, “Mapping human whole-brain structural networks with diffusion MRI,” PLoS ONE, vol. 2, no. 7, p. e597, 2007. [49] P. Hagmann, L. Cammoun, X. Gigandet, R. Meuli, C. J. Honey, V. J. Wedeen, and O. Sporns, “Mapping the structural core of human cerebral cortex,” PLoS Biology, vol. 6, no. 7, p. e159, 07 2008. [50] S. S. Venkatesh, The Theory of Probability: Explorations and Applications. Cambridge University Press, 2012. [51] B. C. Arnold, N. Balakrishnan, and H. N. Nagaraja, A First Course in Order Statistics, John Wiley & Sons, Inc., New York, 1992. [52] W. Hoeffding, “Probability inequalities for sums of bounded random variables,” Journal of the American Statistical Association, vol. 58, no. 301, pp. 13–30, 1963. [53] P. Billingsley, Probability and Measure, 3rd ed. New York, NY: Wiley, 1995.
