Example 2.4K item DNF formula

A disjunctive normal form (DNF) formula is a formula for the disjunction of several terms, each of which is a conjunctive of A Boolean text. KKK term DNF is a DNF formula defined by the disjunctionof KKK term. Each term consists of NNN Boolean characters at most. So, for k = 2k = 2k = 2 and n = 3n = 3n = 3, DNF k item sample for (x1 Sunday afternoon ‾ 2 x Sunday afternoon x3) ∨ (1 x ‾ Sunday afternoon x3) (x_1 Sunday afternoon \ overline x_2 Sunday afternoon x_3) ∨ (\ overline x_1 Sunday afternoon x_3) (x1 Sunday afternoon x2 Sunday afternoon x3) ∨ (x1 Sunday afternoon x3). Is the CCC class of THE KKK term DNF formula learnable by PAC? The cardinality of the class is 3nk3^{nk}3nk, because each term is a conjunction of up to NNN variables, and there are 3n3^n3n such conjunctions, as seen earlier. HHH assumptions set must contain CCC to achieve consistency, therefore ∣ H ∣ 3 or more nk | H | 3 or more ^ {nk} ∣ H ∣ 3 or more nk. Theorem 2.1 gives the following sample complexity bounds:


m p 1 ϵ ( ( l o g 3 ) n k + l o g 1 Delta t. ) . ( 2.12 ) M \ ge \ frac {1} {\ epsilon} \ big ((log3) nk + log \ frac {1} {g} \ big), (2.12)

This is a polynomial. However, it can be shown that the problem of learning K items DNF in RPRPRP is a category of complex problems that allow random polynomial time decision solutions. Therefore, the problem is computationally difficult to solve unless RP=NPRP=NPRP=NP, which is generally not considered to be the case. Thus, although the sample size required to learn the K-term DNF formula is only polynomial, effective learning of such PACs is impossible unless RP=NPRP=NPRP=NP.

Example 2.5k-CNF formula

The conjunctive normal form (CNF) formula is the conjunctive of disjunctive. The k-CNF formula is T1∧… Sunday afternoon TjT_1 Sunday afternoon… Sunday afternoon T_j T1 Sunday afternoon… An expression of the form ∧Tj has j∈Nj∈ Nj∈N of arbitrary length, and each term is a disjunction of up to k Boolean properties. The problem of learning k-CNF formulas can be reduced to the problem of learning links of Boolean characters, which, as mentioned earlier, is a PAC learnable concept class. To do this, you simply associate each item TiT_iTi with a new variable. This can then be done with the following bijection:


a i ( x i ) ∨ . . . ∨ a i ( x n ) – Y a i ( x i ) . . . . . a i ( x n ) . ( 2.13 ) A_i (x_i) ∨… ∨ a_i (x_n) \ rightarrow Y_ {a_i (x_i),… , a_i (x_n}), (2.13)

Where, a_i (x_j) represents the assignment of xix_ixi in TiT_iTi. This simplification of PAC learning for Boolean text links may affect the original distribution, but this is not a problem because in the PAC framework, no assumptions are made about the distribution. Therefore, PAC learnability of Boolean text links means PAC learnability of K-CNF formulas. However, this is a surprising result, since any k term DNF formula can be written as a k-CNF formula. In fact, using associativity, the k term DNF can be rewritten as the k-cnF formula in the following ways:


⋁ i = 1 k a i ( x i ) Sunday afternoon . . . Sunday afternoon a i ( x n ) = ⋀ i 1 . . . . . i k = 1 n a 1 ( x i 1 ) ∨ . . . ∨ a k ( x i k ) . \bigvee ^k_{i=1}a_i(x_i)\wedge… \wedge a_i(x_n)=\bigwedge ^n_{i_1,… ,i_{k}=1}a_1(x_{i_1})\vee… \vee a_k(x_{i_k}).

To illustrate this rewriting in a particular case, look at, for example


( u 1 Sunday afternoon u 2 Sunday afternoon u 3 ) ∨ ( v 1 Sunday afternoon v 2 Sunday afternoon v 3 ) = ⋁ i . j = 1 3 ( u i ∨ v j ) (u_1\wedge u_2\wedge u_3)\vee (v_1\wedge v_2\wedge v_3)=\bigvee ^3_{i,j=1}(u_i\vee v_j)

But, as we saw before, the K term DNF formula is not effectively PAC learnable! What explains this apparent discrepancy? It is observed that the number of new variables required to write the k term DNF into the k-CNF formula by the transformation just described is exponential in K, O(nk)O(NK)O(NK)O(NK). The difference comes from the size of the concept representation. The K term DNF formula can be an exponentially more compact representation, and effective PAC learning is difficult to handle if polynomial time complexity of this size is required. Thus, this apparent paradox relates to key aspects of PAC learning, including the cost of concept representation and the choice of hypothesis sets.