A Comparison of Markov Based Logistic Model Determining the Risk Factors of Health Conditions for Old Aged American People By ASEK MD. SUZAUDDIN
Abstract:
Markov model are expedient and very serviceable method for analyzing longitudinal or categorical data. This method play an important role in various fields to explain the dependence pattern of a time series over a period of time and to predict the future course of process behavior by its investigative and prognostic power.
Health and Retirement Survey (HRS) data, an ongoing longitudinal survey that interviews a national sample of persons born between the years 1931 and 1941at two year intervalsWe discussed two discrete time markov models proposed by Muenz-Rubinstein and Azzalini respectively for the HRS dataset and fit this two models considering health conditions as a dependent variable. We also estimate the efficiency for the two fitted model and compare the models on the basis of the results.
This study fits the two models and compare them on the basis of their efficiencies. In here we fit those models for the Health conditions of the old peoples and obtain the efficiencies for both models and found that the estimate of Azzalini’s model is more efficient than that of Muenz-Rubinstein model.

1. Introduction
Longitudinal data are defined to be repeated measurements on sampling units (typically, individuals) over time.The longitudinal studies or follow up studies are repeated over an extended period of time in order to measure the rate and degree of change occurring in patterns of response. In longitudinal studies, information of different subjects are collected at different time points as they are observed in relation to an event of interest for a specified period of time. The outcome variable and a set of covariates which are observed repeatedly characterize the longitudinal data for each subject who belongs to a population under study. Ideal longitudinal measurement would record health and other characteristics continuously, accurately, precisely, and completely over a time interval of interest. Measurement may be inaccurate, imprecise, or subject to changes over time, which may lead to biased assessment of association. Data rarely are completely ascertained in prospective observational studies, and depending on the mechanism that underlies data being incomplete, naively treating their incompleteness may severely distort findings on strength of associations and sabotage the ability to isolate associations to hypothesized causal factors.
A number of statistical methods are available which are used to analyze categorical data with a broad scientific objective to describe the effect of certain risk factors on the evolution of event of interest. The objective of statistical modeling is to study the relationship between a dependent variable and some independent variables. This relationship can be used to identify the risk and prognostic factors of a specific disease as well as to predict the future disease status.
Multi-state models are commonly used in studies of chronic disease, in which patients are assumed to pass through a series. Markov models are convenient and useful method of estimating transition rates between levels of categorical response variables, such as disease stages, which changes over time. Markov models are appropriate models for certain time series in which the observation at a given time are the category into which an individual falls. In recent years, Markov chain models are used in various fields because of its interesting properties to model longitudinal data and its exploratory power.
Background of the study
Markov models are often used for its analytical and predictive power in recent times. So far, a number of attempts are made to develop Markov model to cope with different situation by many researchers.
Methods of the estimation of Markov models parameters are well established and relatively easy to fit, particularly if time homogeneity can be assumed. Time homogeneous Markov models have been used in a number of applications. A quasi-likelihood algorithm was proposed by Kalbfleisch and Lawless (1985) using Markov models to smoking prevention study. They used the models to panel data in which individuals are viewed over only a portion of their life history and complete information about the transition times between states are unavailable.
Richard Kay (1986) applied this model for analyzing cancer Markers and disease status in survival studies. He proposed a Markov model for intercommunicating states for modeling relationship between survival time and disease status
Longing et al. (1989) have applied the semi multi-state Markov model with one-way forward transition successfully to the stages of HIV infection. But they have found the transition probabilities using a different technique from that suggested by Kay (1986). The model also includes no covariates, and is fitted to heavily censored data.
Gentlemen et al. (1994) applied such models for HIV disease. They also used multi-state models for analyzing incomplete disease data and applied the general estimation methods of Kalbfleisch and Lawless (1985). They ignored the issue of covariates for simplicity. Gladman et al. (1995) applied such models to identify markers for severe disease in Psoriatic Arthritis (Ps A).
P. Saint-Pierre et al. (2003) applied such models to the analysis of asthma control with use of covariates.
Regier (1968) re parameterized the two state transition matrix to include a parameter which is the ratio of odds of staying in state zero to the odds of staying in state one. This can detect the tendency of migration from one state to another. Here zero and one denote the two possible states of the Markov chain.
Prentice and Glockler (1978) proposed a grouped data version of the proportional hazards regression models for estimating computationally feasible estimators of the relative risk function and the corresponding survivorship function in the presence of many tied failure times.
Korn and Whittemore (1979) modeled the probability of occupying the current state at the previous time point as one of the covariates.
Wu and Ware (1979) proposed a model which include accumulate covariate information as time passes before the event and considered occurrence or non-occurrence of the event under study during each interval of follow up as the dependent variable. The method could be used with any regression function such as multiple logistic function and that suggested by Prentice and Gloeckler (1978).
In 1985, Muenz and Rubinstein proposed a discrete time Markov chain for modeling covariate dependence of binary sequences. They denoted no distress by zero and distress by one. At each time point and individual may be in state zero or one.
The transition probabilities of the type zero to zero and one to zero were modeled by two separate logistic regressions showing how the covariates relate to change in state. With P covariates, there are 2(p+1) parameters including intercepts, which are estimated by maximum likelihood method. They also gave an extension of the basic model, which allow time dependent covariates or non-stationary or second order Markov chain.
Some other Markov chain for analyzing repeated categorical and ordinal data proposed by Ware, Lipsitz and Speizer (1988), Bishop, Fienberg and Holland (1975), Chuang and Francom (1986) , Hooper and Young (1988) consider multistate Markov models with states denoting levels of the observed outcomes.
Albert (1994) proposed a finite Markov chain model for analyzing sequence of ordinal data from a relapsing-remitting disease. Albert and Myron (1998) developed a class of quasi-likelihood models for a two state Markov chain with stationary transition probabilities for heterogeneous transition data.
Azzalini (1994) introduced a stochastic model, more specifically, a first order Markov model, to examine the influence of time dependent covariates on the marginal distribution of the binary outcome variables in serially correlated binary data.

In this paper an attempt is made to fit the discrete-time Markov models proposed by Muenz-Rubinstein and Azzalini respectively for the covariate dependence of binary sequences taking Health Conditions as dependent variable. And after that we obtain efficiency for the both models to compare on the basis of the results. In our study we take only three independent variables and all computations have done in R-plus.

2. Data and Variable:
Data Description:
To illustrate the overall health condition of the old American and to apply the methods used in this and subsequent chapters, we have made use of Health and Retirement Survey (HRS) data. this dataset collected by RAND Center for the study of aging, with funding and support from the National Institute on Aging (NIA) and the Social Security Administration (SSA).We have made use of Health and Retirement Survey (HRS) data to fit discrete-time Markov model. In this chapter we will describe the data used and the related variables considered.
HRS Data
In this study we have made use of health and Retirement Survey (HRS) data in order to fit discrete-time Markov models. We also have made use this HRS data to describe the overall health conditions of the old American people. The health and Retirement Study (HRS) is conducted by the institute for Social Research (ISR) at the University of Michigan in Ann Arbor and supported by the National Institute of Aging (NIA).The study interviewed 30,405 Americans aged 50 and over every two years on subjects like health care, housing, assets pensions, employment and disability. The HRS data are available for downloading by researchers and analysts at no cost on the website http://hrsonline.isr.umich.edu.
The study is managed through a cooperative agreement (NIA U01AG009740) between the NIA, which provided primary funding, and the ISR, which administered and conducted the survey. The main goal of this survey was to provide panel data that enable research and analysis in support of policies in retirement, health insurance, saving and economic well being.
Respondents in the initial cohort were those who born during 1931 to 1941.This cohort was first interviewed in 1992 and subsequently every two years. A lot of 13434 respondents were included in this cohort. The panel data documented by the RAND, from the HRS cohort of nine rounds of the study conducted in 1992(Wave 1), 1994(Wave 2), 1996(wave 3), 1998(Wave 4), 2000 (Wave 5), 2002 (Wave 6), 2004 (Wave 7), 2006 (Wave 3) and 2008 (Wave 2008) used in this study for application and comparative purposes.
Variable:
Dependent Variable
Health Conditions (RwCONDE)
One health problem index, RWCONDE that is the sum of indicators for whether a doctor has ever told the respondent that he or she has ever had a particular disease, has been derived. The eight included diseases are high blood pressure, diabetes, cancer, lung disease, heart disease, stroke, psychiatric problems, and arthritis. The values of this variable range from zero to seven (0 to 7). The value zero (0) indicates the person has no disease while any value between 1 and 7 indicates the person has at least any kind of disease. Finally, the variable has been categories into two categories namely “No disease” for the value zero (0) and “Have any disease” for the values between 1 and 7.That is categorize as Health Conditions =[pic]

Independent variables:
Here we’ve used three variables, Gender, Age and BMI (Body Mass Index) in our study.

Age Variable:
Age variable is a continuous time dependent variable. In the HRS data file age variable is not given directly but given in days, months and years. In our study we’ve considered age in years. We’ve computed the age from the data set. It is mentioned before that we’ve used only first four wave that is the HRS cohort which is consists of people born between 1931 to 1941 was first interviewed in 1992 is as first wave(w1),in 1994 is second wave(W2),in 1996 is third wave(W3) and in 1998 is the fourth wave(W4).The variable has been created by subtracting the birth year from their interviewed year that is 1992 for W1,1994 for W2,1996 for W3 and 1998 for W4.Therefore the age varies from 51 to 67.Finally,the age variable is recoded into two categories: Age 60 and less than 60 for the value zero(0) and Age over 60 for the value one(1).We categorize as follows Age = [pic]
Gender:
The variable Gender is a categorical variable of the respondents having the categories ‘1 for male’ and ‘2 for female’ in the original data set. In our study, we consider ‘0 for female’ and ‘1 for male’, we further categorize gender as Gender = [pic]

BMI:
Body Mass Index (BMI) is also a continuous time dependent variable which we can obtained as dividing the weight of the respondent by the square of their height, where height is converted into meters and weight into kilograms. For the purpose of the study we considered: ‘0 for age below 18.5’,’1 for age 18.5 to 24.9’,’2 for age 25.0 to 29.9’ and ‘3 for age 30 and over’. The categories are as follows: BMI = [pic]

χ2 –Test
In this chapter we are interested to examine the association, if any, of the health condition of American people with several characteristics. This work will be performed through chi-square test of independence.
In this case the test statistic is, χ2 = ∑ (Оi - Еi )2 / Ei
We can conclude whether the variable is associated or not, by the help of the value of this test statistic and the P-value. P-value is the lowest level of significance for which we can easily reject the null hypothesis.

Table 4.2 Test of Significance of Health Condition with Different Factors:

|Variables |Categories |Health Condition |χ2 |
| | | No Disease |Have Disease | |
|Gender |Male |5437 |11943 |29.31** |
| |Female |5902 |11453 | |
|Note: P t) = Pr (Yit / Yit(1)

The transition probabilities po = pit, o = Pr (Yit=1 / Yit(1 = 0) and 1- po = Pr (Yit =0/ Yit(1 = 0) p1 = pit, 1 = Pr (Yit =1/ Yit(1 = 1) and 1- p1 = Pr (Yit =0/ Yit(1 = 1)

define the Markov process but do not directly parameterize the marginal mean. Azzalini (1994) parameterizes the transition probabilities through two assumptions. First, a marginal mean regression model is adopted that constrains the transition probabilities to satisfy
(it = pit, 1 (it-1 + pit, o (1( (it-1) (2)

Second, the transition probabilities are structured through assumptions on the pair wise odds ratio. (it =[pic] (3)
Which quantifies the strength of serial correlation. The simplest dependence model assumes a time homogeneous association, (it =(o; however, models that allow (it to depend on covariates or to depend on time are possible.

Solving (2)and (3) for po and p1 for any t>1.
First we consider t = 2 then (2 = (1p1 + (1-(1)po or, po = [pic] and 1 - po =[pic]
Now, if ( =1, then we have po = p1.

and if ( #1, then putting the values of po and 1-po we have

( = [pic]

4. Results:

Muenz-Rubinstein Two State First Order Markov Model :
Here, we illustrate the process to fit the model.
Table 1 shows the transition count during consecutive follow-ups in the period 1992-1998.

Table 1: Transition counts
| | Transition Counts | |
|States | |Total |
| | 0 | 1 | |
| 0 | 7111 | 1553 | 8664 |
| 1 | 202 | 15627 | 15829 |

Estimating Parameter :
Table 2 shows the parameter estimates from the model. In here, we see the independent variables considered have significant association with the health condition index.

Table 2 : Estimates of parameters of two state first order markov model for health conditions Transition 0 to 0

|Variable name |Co-efficient |Standard Error | Wald w | p-Value |
| Constant | 2.0217 | 0.08476 | 23.851 | .000 |
| BMI | - 0.2090 | 0.03721 | - 5.6165 | .0006 |
| Gender | - 0.1179 | 0.05689 | - 2.0741 | .5152 |
| Age | - 0.1186 | 0.05680 | - 2.0891 | .4965 |

Transition 1 to 0

|Variable name |Co-efficient |Standard Error | Wald w | p-Value |
| Constant | - 5.6639 | 0.0691 | - 81.926 | .000 |
| BMI | 0.6683 | 0.0264 | 25.270 | .000 |
| Gender | 0.4044 | 0.0407 | 9.925 | .000 |
| Age | 0.4160 | 0.0418 | 9.929 | .000 |

For transition 0 to 0 the variables BMI, gender and age have a negative association and For transition 1 to 0 the variables BMI, gender and age all show a positive association with p-Value zero.

Azzalini’s model for covariate dependence of binary sequences:

Parameters Estimate:
The binary response of interest can be defined as
[pic]
where t= 1,2,3,4 The response Yet to be generated by a binary Markov chain taking value 0 and 1 with the transition probabilities [pic]for j = 0, 1 and t = 1,2,3,4.

Table 3: Estimate of parameters for the Azzalini’s model to the HRS data.

|Variable | |Estimated |
| | |coefficient |
| | | Muenz-Rubinstein |For the transition |For the transition |
| |Azzalini | |0 to 0 |1 to 0 |
| | |For 0 to 0 |For 1 to 0 | | |
|BMI |0.00378 |0.06895 |0.12813 |0.0548 or 5.4% |0.0295 or 2.9% |
|GENDER |0.00564 |0.00323 |0.08269 |0.5726 or 57.2% |0.0682 or 6.8% |
|AGE |0.00710 |0.1606 |0.08738 |0.0442 or 4.4% |0.0812 or 8.1% |

In this study, we have seen that all the estimated parameters for Azzalini model are more efficient than that of Muenz-Rubinstein model for two types of transitions 0 to 0 and 1 to 0 except the Gender of 0 to 0 transitions. The estimated value, for 0 to 0 transitions, of the co-variate Gender for Muenz-Rubinstein model shows approximately 57% more efficient than that of Azzalini’s model. But in all other cases, it is clear that the estimates of Azzalini are more efficient than Muenz-Rubinstein model.

5. Discussion:
To conclude the study we will first discuss the bi-variate analysis and the percentage distribution so that we can interpret the results well. And then we conclude for the fitted discrete- time Markov models for the HRS dataset.
In this study, 50.3% are male and of them about 68.7% faces disease while of the women approximately, 66% are face any disease. Here, having diseases are more in male people than that of the female people. Therefore, we can say that the focus should be given to the male regarding health related problems so that they can live without any disease.
This study represents how ages affect the health conditions to the people in USA as people are being affected as long as their ages are increasing. The study reveals 70.5% of people aged over 60 are facing different diseases while this percentage is relatively low for people aged up to 60 and it is about 64% as well as it countenances the generalized essence of being more affected as long as the ages increase.
The study reveals, people having difficulties are more confronting the health condition in last age.
Analysis of longitudinal data is an important part of the statistical analysis that enables us to identify the risk factors and prognostic factors and to explain the effect of the factors in determining the status of the outcome variable. So far, large number statistical models have been suggested by several researchers. Not only may the covariates influence the current behavior of the process also have important contribution determining the present behavior of the process. For such processes usual statistical models cannot explain the complete dependence pattern of the outcome variable. In these cases , in such situations, Markov modeling is one of the best solutions. In this study we fit both discrete time Markov models proposed by Muenz-Rubinstein (1985) and Azzalini (1994) respectively as a multivariate analysis. After that we also find out the efficiency and compare for the fitted models. From this above result we may conclude that almost all variables except one, Azzalini’s model is more efficient than Muenz-Rubinstein model.
Finally, this study is not an exhaustive evolution. As the Markov model is frequently used to explain the dependence pattern of a time series over a period of time and to predict the future course of process behavior by its analytical and predictive power, more attention should be given to estimate the parameters and to develop new statistics for this purpose.
However, the problems of the fitting model open the opportunity for the researchers to work on this field.

References:
Agresti A. Categorical Data Analysis (2nd edn): New York, 2002.
Albert, P. S (1994), A Markov Model for Sequence of Ordinal Data from a Relapsing Remitting Disease. Biometrics, 50, 51-60.
Albert , P. S. and Myron, A. W. (1998), A Two State Markov Chain form Heterogeneous Transitional Data: A Quasi likelihood Approach, Statistics in Medicine, 17, 1481-1493.
Anderson, T. W. and Goodman, L.A. (1957) Statistical Inference about Markov Chains.
Annals of Mathematical Statistics, 28:29-110
Chatfield C. (1973). Statistical Inference Regarding Markov Chain Models. Applied Statistics; 22: 7-20
Gentleman RC, Lawless JF, Lindsey JC, Yan P. (1994). Multi-state Markov models for analyzing incomplete disease data with illustrations for HIV disease. Statistics in Medicine; 13:805-821.
Gladman DD, Farewell VT, Nadeau C. (1995). Clinical indicators of progression in psoriatic arthritis: multivariate relative risk model. Journal of Rheumatology; 22:675-679.
Goodman L.A.(1958a), Simplified Runs Tests and likelihood Ratio Tests for Markoff Chains. Biometrika : 45, 191-197.
Health and Retirement Study, (Wave [1-7]/Year [1992-2004] public use dataset. Produced and distributed by the University of Michigan with funding from the National Institute on Aging (Grant Number NIA U01AG09740). Ann Arbor, MI.
Islam, M.A. and Chowdhury, R. I. (2006). A Higher-Order Markov Model for Analyzing Covariate Dependence. Applied Mathematical Modelling, 30:477-488.
Islam, M.A. and Chowdhury, R.I. (2008). Chapter 4: First and Higher Order Transition Models with Covariate Dependence. In Progress in Applied Mathematical Modelling, F.Yang (ed), Nova Science , New York, 153-196.
Islam, M.A. and Chowdhury, R.I. and Huda, S. (2009). Markov with Covariate Dependence for Repeated Measures. Nova Science Publishers, Inc. New York.
Kalbfleisch JD, Lawless JF (1985). The analysis of panel data under a Markov assumption. Journal of the American Statistical Association: 80:863-871.
Kay R. (1986). A Markov model for analyzing cancer markers and diseases states in survival studies. Biometrics: 42:855-865.
Korn, E.L.and Whittemore, A. A. (1979). Methods of Analyzing Panel Studies of Acute Health Effects of Air Pollution. Biometrics, 35, 795-802
Muenz, L. R. and Rubinstein, L.V. (1985). Markov Models for Covariate Dependence of Binary Sequences, Biometrics, 41, 91-101.
Reeves G.G. (1993). Goodness-of-Fit Tests in Two-State Processes. Biometrika 80: 431-442.
Regier, M.H. (1968). A Two State Markov Model for Behavior Change. Journal of the American Statistical Association, 63, 993-999.
Wu, M.and Ware, J. H. (1979). On the Use of Repeated Measurements in Regression Analysis with Dichotomous Reponses. Biometrics, 35, 513-522.
A. Azzalini (Logistic Regression for Autocorrelahed data With Application to Repeated Measures)-1994.
Adrian Raftery; Simon Tavare (Estimation and Modeling Repeated Patterns in High Order Markov Chains With the Mixture Transition Distribution Model)-1994.

Author’s addresses: